Making the web easily acccessible through chat bots. Created in 24 hours at tamuhack 2020.
Winner of both Microsoft’s and Gartner’s Sponsor Challenge at TAMUhack 2020.
After using a speech to text interface, we realized it would be a cumbersome way for visually impaired people to browse the web. We decided to make a chatbot that could execute and interface with the web on behalf of the user and could be easily implemented by developers for their own sites.
Our chrome extension-enabled bot gathers input from the user and uses machine learning and NLP to associate user input with actions that the bot can execute. The extension is also programmable for developers to customize it to each website, so they can add unique, higher level actions to improve their specific user experience. The program then executes the actions on the webpage and gives feedback to the user.
First, we would like implement text-to-speech methods to read our bot’s messages to the user. In addition, we would like to develop higher level functionality for the chatbot, with deeper and more complex capabilities on a wide variety of sites.
Because of this, we tried to design Helping-Eye in such a way that developers could add to its functionality in the future. A future for Helping-Eye or something as similar as an open source resource would be tremendously beneficial for visually impaired people, and everybody else.