Mouse echo TTS app using Windows OCR + UI Automation
$30-250 USD
Cancelled
Posted over 7 years ago
$30-250 USD
Paid on delivery
0) The main window should have two checkboxes: "Use OCR" and "Use UI Automation".
Below each checkbox should be a dropdown list that the user can use to select a voice for each choice from the list of installed SAPI voices. There are two voices installed with US Windows 10 (Microsoft David desktop and Microsoft Zira desktop), so default to using those two. If the user changes checkboxes or the selected voice, save their settings to appropriate user app data file.
Three buttons: Save, Cancel, Exit
App should honor the Speech speed in the Speech control panel.
That's all the UI we'll need for this app.
Two functionality milestones:
1) Windows 10 includes a free OCR module. See [login to view URL]
You can also get OCR for Windows 8.1 store apps: [login to view URL]
[login to view URL]
I need an app for moderate-vision-impaired people, which speaks whatever text is under the mouse, using TTS. So basically, if the Use OCR checkbox is checked:
- on mouse hover for one second, take a snapshot of the screen
- Run OCR on the contents of the screen.
- Figure out what text object is under the mouse pointer when you took the snapshot
- flush the TTS speech output queue
- Speak that text element (largest appropriate size returned)
2) Look at the Inspect tool from MSDN: [login to view URL](v=vs.85).aspx
[login to view URL] does something kinda similar: if you hover the mouse over any item, it returns a bunch of information. Typically the Name attribute of the information returned corresponds to the text shown on the screen, but not always. Sometimes the Name is wrong or missing (that's why we need the Mouse Echo feature). Sometimes the Name is a more complete version of what is visible. Sometimes the Name is a label, and the text you're hovering over is the Value. So if the Use UI Automation checkbox is checked:
- on mouse hover for 1 second, get the UIA object under the mouse
- flush the speech queue
- speak the Name, Role, and Value using the TTS voice they selected
So if both checkboxes are checked, we'll here the text first, then (typically in a different voice) the accessible item, which might include the same text.