Welcome to my project which is about books , paper and education hacking with computer vision. Throughout the years it seems that everyone started dropping paper books and switched to electronic versions, I know its good for trees and what not but still reading a book in your hand while sitting on that old arm chair has a feeling that cannot be compared with staring at a computer screen like a zombie.
On January 2014 I was invited to the mother ship @IOActive for a snowboarding trip at mount baker it has inspired me to come up with this idea because during my stay there when the sun sets it goes completely dark. When we arrived at our cabins first day we were handed some cool swag that had a headlamp so you can walk back and forth between your cabin and the party cabin and get lost on either way which was cool.
After this trip I returned back to my country where the lights also cut in my neighborhood every often, suddenly it all came together and an idea struck me like a lightning bolt so I said to myself what if I can revive book reading and make it cool again. I’ve put my lab coat on and started building a contraption which yielded the following:
By doing that I thought I was onto something which appeared to me as an awesome concept .. Lets build a research assistant I said. The ideas started to flow through my brain.
Phase I (light conditions):
- Check if night reading is possible with the webcam under certain conditions which was possible when adding a filter to the headlamp.
Phase II (what do i need this to do):
• A computer vision reading , learning assistant and miniature researcher.
• No privacy concerns.
• A pluggable framework for computer vision enthusiasts and programmers.
• A new way to encourage reading & learning.
• Hand gestures to allow you to copy text, image search , create a book summary, enable static analysis for code reviews, solve mathematical equations.
• Create a virtual drawing board for notes
• Interacts you with people via voice chat that are reading the same book. A social network based on learning.
• Create summaries of your books.
• Allow it to create course materials for the subject you are reading depending on your expertise level (beginner , intermediate , Advanced)
• Spell Checker and language translator.
• Monitors page flips against time consumption for each page you read.
• Optical character recognition engine.
• Learns what YOU teach it (The best way to learn is to actually teach).
• Speech to text and voice commands ability. (I used speakable commands , Dictation , and Automator thanks to my Mac)
Phase III (Gimp & PowerPoint the concept):
And now to jolt down some gestures that would be useful.
• Book Lookup
• Start static analysis security code review assistant
• Copy text ( single line/word or multi-lines)
• Join Reading room
• Create learning material( search for online courses , pdf’s ,blogs , wiki ,forms…..etc)
• Share anything with (tutor, Twitter, Facebook, online book reading room, class mates .. Etc)
• Internet search(image, text, equation lookup/ solution, symbol, … Etc)
• Research (after a line/multi-line/image selection it will crawl the internet for such topic and create folders with downloaded files/links for your next phase of learning even if its math equations you don’t understand)
• Highlighting (remembering page and book of highlight)
• Finger Drawing (to take side notes without damaging the book)
• Record Video , take a picture of a page.
• Start Mic session to record your notes and control the computer for something like opening a new word document and dictating via speech.
• Etc (limitless possibilities)
Phase IV (Futuristic ideas):
• Automatic rotatable camera
• Can flip a page and learn for you
• ECG sensors to present your brain activity and let you know if you are relaxed or tense.
Now that we have mostly everything needed to start bringing this to life I had to learn a few things which were programming C++ and understanding OpenCV.
In the next post below I’ve made a gearing up section for people wanting to learn C++ and openCV by adding links to tutorials , books and configurations to everything i have learned during this project.
Without further adieu I present you
EDUCATION LENS (EDULENS)