NYC IoT Hackathon recap: Apps to help with justice, the visually-impaired, and shopping

On November 18th and 19^th, we flew across the country to Manhattan, New York for one of our final two IoT hackathons of the year as part of the AT&T Hackathon Invitational. We had a truly incredible response with 794 registrations resulting in over 200 hackers coding up to 33 apps. This group of developers truly deserve every accolade possible as they worked through some truly tough Wi-Fi experiences to produce their hacks.

An extremely special thank you goes to  Nuance, for not only sponsoring, but they also provided some amazing assistance at troubleshooting the onsite Wi-Fi. Two amazing network engineers from their team spent hours both Friday night and Saturday morning and replaced the venue’s router with a custom-built solution. I’d also like to thank our sponsors at Intel, IBM, and Twilio for their support of the event and how they assisted teams with their hacks.

Our big winner was Interview IQ, created an intelligent interview platform that provides important feedback to the candidate and the interviewer. The app leveraged Twilio, Intel, and IBM Watson.

Another interesting app idea was In-n-Out, which created a platform for shoppers to avoid long lines at the register using BLE technology to track purchases, and fast-track them through the shopping experience.

Now onto the winners:

• 1st Place Best App Overall – Interview IQ
• Best Mechatronics Solution Overall – PillTime
• 1st Place Best Use of Nuance – PillTime
• Honorary Mention Best Use of Nuance – Photobot
• 1st Place Best Use of IBM Bluemix – Ultima
• 2nd Place Best Use of IBM Bluemix – Pavio Care
• 1st Place Best Use of Intel Technology – Home Sweet Home
• 1st Place Best Use of Twilio – Smart Plant

Check below to find out about all the apps created at the hackathon (please note all descriptions are provided by each team):

PillTime– PillTime will help you keep track of your pills and medications. The system allows the user to create a simple schedule to remind them to take their pills with the use of voice commands. The user simply puts the pills in one of the designated pillboxes and speaks to the  application to notify PillTime when the pill needs to be consumed and where it is located. This command is the last interaction the  user has to set up. The  user can go about their day without any stress regarding their medication. When the time comes to take their medication, the user will receive a text message using Twilio to alert them of their medication time. Also, a light will be illuminated on the box to avoid confusion with the clutter of pill bottles, which can help reduce incorrect pill consumption. If the user puts off taking their pills, the PillTime will send a phone call an hour after the medication time to remind the user. That is what we have produced in the past 24 hours.”

Mocho – “Mocho is a medical translation app. Many doctor and patient interactions which involve voice are misunderstood by patients. The app (and a kit) uses Nuance Mix to return a transcript of the speech, get keywords, speak in the patient’s native language, and show video.”

Blockfinity – “Ultima is like Siri for emergency situations; it is a virtual assistant for emergencies. For example, during a fire it would give the current temperature and contact family members to check safety. It’s the true personal risk adviser, providing guidance through augmented knowledge DURING the emergency situations, and accountability in the AFTERmath of a crisis.

The app detects emergencies nearby using social media data, sensors, IoT, user reports, and private, public and emergency networks. These large data sets are quickly processed using quantum computers. Currently, the IBM Quantum Computer is being used. This allows for the user to receive important relevant information in real-time. This can help prevent injuries and potentially save lives and property.”

Telmi – “It’s an educational device to help kids or people with poor vision recognize things.”

Appliedat – “Effortlessly save job listings and access them later from virtually anywhere.”

Smart Plant – “Making plants smart again.”

iStroller – “A smart baby strollers tracking mechanism that are notified of the weather changes, so you can better keep the baby warm, safe, and alert.”

Drone Master – “Drone Master allows users to remotely control drones to collect data. Whether it is network heat map data, pollution, or in this sample, noise pollution, the user can direct the drone to provide real-time data. Utilizing simple go-to commands using the Mapquest and Google API’s, a user can easily control simulated drones with the Dronesmith API. They can also communicate with the drone through Twilio, getting real-time feedback on the drone’s location and current data stream.”

Digital Doors for Smart Living (Homes) – “Digital Doors provides the next generation in home monitoring. We can determine if a cat or your grandmother went through a doorway. We use machine learning keep information to determine the wellbeing of your love ones. Voice communication is established to notify events or query activity.”

Robotic Arm_Iot – Hack – “Intel Nunc Server/Arduino 101 Grove shield interfacing of 4 DC motor axes of a Robotic Arm that are wirelessly controlled via a web app.”

In-N-Out (I-N-O) – “In-N-Out removes the pain for shoppers who have to wait in lines before checking out of the physical store. Using the iBeacon-enabled Intel Edison technology, In-N-Out creates a truly seamless shopping and checkout experience for shoppers.

As shoppers walk into the store, our app verifies their bank accounts. Items that shoppers pick up and that are in close proximity to their mobile devices, which are connected to the Intel Edison as an iBeacon in the store, are automatically added to their shopping cart. Shoppers can also easily view product information and make a better decision with their purchases. Then, payment and checkout will be completed once shoppers walk out of the store. In-N-Out truly allows users to get in and out of the store in a flash and avoid having to wait in line to complete their transactions.”

Pavio Care – “Pavio Care is a patient-centric mobile and IoT application that enables healthcare providers to help reduce preventable re-admissions for at-risk Diabetes patients. Hospitals require access to real-time and predictive data so that they can make fast and effective decisions. Pavio Care through its innovative platform provides scale for physicians and nurses to manage a large pool of patients.

With its ability to monitor patients’ vitals, it has the capability to take intelligent and cognitive prescriptive measures to help address any threshold breaches. Pavio Care has the ability to access nearby Telehealth social workers for any emergencies. Its unique gamification and localization approach to learning will improve patient’s retention and adherence of the transition care procedures.”

iBobs – “Virtual Assistant for the beauty industry. iBobs completes tasks while stylist do their clients! A verbal response platform that creates client files with contact info, color formulas, and appointment notes and puts it into a paperless file. iBobs handles appointment services. It can help decrease wait time between clients with ETA auto response. Merging beauty with technology can help save time, money, and the environment.”

Dexterous – “The most frustrating part of shopping is the part where we stand in queues for buying our purchases.

Our app Dexterous is intended to help solve this problem and make shopping a hassle-free process. Retail companies spend nearly $10 billion on energy each year. Our app intends to provide at least a 10 percent reduction in energy utilization without affecting the customer service. Our App Dexterous helps the store managers to continuously keep track of the number of customers in the store at any point of time using the heat sensors present at various points in store. When the values generated by sensors reach a threshold, Dexterous sends a notification to the store manager with recommendations on improving customer service such as activating new tills, and adding more self-checkout counters. Furthermore, our app will also have the feature to automatically enable and disable the self-checkout counters based on the crowd in store. This app helps in effective utilization of energy and in improving productivity of employees.

Furthermore, this app can also help the store manager to analyze the peak times for the store and aid in redirecting employees to certain sections of stores during peak hours. This app will also help store managers to analyze most frequently visited areas of the store and help in product placement.”

Home Sweet Home –“Matchbox-size battery operated network scanner, which notifies you when your loved ones arrive home.”

SmartBin – “SmartBin is an IoT trash bin that revolutionizes how trash is handled, providing unique functionality that automates the recycling process and allows cities and organizations to understand people’s habits with data from a whole new perspective.

SmartBin automatically separates all incoming trash using IR sensors and computer vision into three categories (following NYC’s recycling standards): plastic/metal, paper, and general trash, effectively automating separation of trash. If a SmartBin is becoming close too full or a suspicious object is inserted (like a gun, for example) it sends the responsible party a SMS message using Twilio.”

FiveOnPoint – “It’s a matching app that provides insights into personality traits given the introductions of each user. There is also added functionality for phone calls between accepted matches, so that there can be further updates of these personality traits given phone conversation. This gives the users a better sense of the personality of their matches before they decide to meet in real life.”

Izzy – “Izzy is a wearable IoT-connected ‘Don’t Panic Button.’ In case there’s an emergency, the person clicks the Izzy button and their loved ones are sent a text message alerting them of a potential problem. Their GPS coordinates are logged as well as a 30-second audio sample. The loved ones can then make a decision about what to do next.

Power conservation is super important and Izzy can last on standby for up to weeks. It uses the AT&T wireless network via a Particle electron. This would be a great tool for parents with kids, adults with elderly parents, single women, and anyone involved in high-risk sports like rock climbing and mountain biking.”

Boing – “Imagine that you’re sitting comfortably at home right now playing with your phone. Suddenly, you receive a text that someone has knocked on your door. That is the essence of Boing technologies (no, not the planes). It connects everyday sounds such as knocking or your front door and closing with everyday actions to save the user both time and money.

How our product works: The user first sets action through a recording, for example, a double knock will send a text to your phone. Or when the front door closes, all the lights in your home will turn off. Our product uses the Raspberry Pi audio finger printing technology to match sounds and an Android application to give the user the flexibility to specify what kinds of actions are wanted.

So here we are, we created a technology that simplifies everyday routines through everyday sounds that we make.”

blind.eye – “Give back power to visual impaired human beings. Implemented as an iOS app, it identifies all the objects in front of a person and tells it to the person so that they are aware of their environment. This will help them navigate better in their environment. They will also be able to tell if the human being in front of them is a man/woman or their approximate age.

Uses: IBM Watson, Google Tensorflow, Nuance for Speech, (Reach: Twillo for alerting the relatives)

Reach Features: When they say help me to the phone, alert relatives, and/or friends!”

See it – Tell it – a Hotline ChatBot – “See it – Tell it will create a platform to keep all Americans safe and empower authorities of all levels to solve or prevent criminal and or terrorist activity. It will create a platform for taking the fear out of terror and help to solve crimes. See it – Tell it will help ensure the Federal Bureau of Investigation (FBI), Homeland Security, CIA, or local police will be able to access reports with information on people reporting people, and events that are suspicious. Using the app, online, phone (speech-to-text), and kiosk you will be able to access the form to file a report. This will help eliminate the frustration of trying to reach the correct authority and give a base operation to find information.

The strategy for implementation has been well thought out and will work as a tool for reporting, analyzing, and keeping track of the people who have acted suspiciously. It will create a system that people all over the world can use to be able to easily report suspicious activity, people, or any other terror like issue.

Since 1995, there have been multiple The one thing these attacks have in common is someone – a friend, family member, or a police officer had stopped them for an offense – yet the attack was not stopped. How many tips etc. came in after the fact? If someone had a way to report this suspicious activity, the event might have been prevented.

See it – Tell it uses tone analyzer, speech-to-text (transcription), Alchemy text analytics, sentiment analysis, personality insights, and eye tracking technology for face time or kiosk reporting. The security of the reporter will be of the utmost importance, and this technology will be a great assist for someone who is held in silence against their will. This will provide a universal/worldwide way for people to report and prevent tragic events.”

Photobot – “Photobot is a Raspberry Pi bot that understands intents and can take photos based on voice commands. It can also send photos to your phone automatically using Twilio’s messaging API.”

Juror 13 – “We created a platform that ensures that the crime matches the time. We take color out of the law and ensure that mass incarnation for minorities is lowered by using Watson to make recommendations on sentencing ensuring checks and balance in our judicial system. Minorities served 50 percent more time for the same crime as non-minorities. Our platform can disrupt the antiquated judicial system using technology. We use Bluemix as a platform to link Watson, Nuance, and Twilio to create a solution that can help ensure and display fair law practices.

The judicial system is designed for the protection of the rights and liberties of all American citizens. However, racial discrimination has permeated the judicial system for decades. This very fact leads me to believe that we must challenge the ideal that men and women alone should be charged with the protection of one’s right to life, liberty, and the pursuit of happiness. Juror13 uses Artificial Intelligence (AI) to assist or inform all parties involved in the trial with findings and delivering an un-biased verdict or prediction (whatever the developer is able to build). The platform or app is not intended to replace due process or human participation, but aims to eliminate and/or reduce bias in the judicial system and promote transparency, fairness, and justice, for all. The app or platform assists all parties involved in the court; from the juror to the judge and will ultimately build the capacity of the justice field to combat structural racism across the nation.”

Project Orcana – “A mobile app to help anxiety and stress that accompanies mental therapy.”

Bodmon – “Coming from a development and a neuroscience background, we have engineered an end-to-end solution for collecting and analyzing daily body movements to aid physical therapists in optimizing patient treatments during therapy sessions. Doctors can monitor unstable movements, bad posture, and patient progress before a patient visit.”

Sign On – “Our application records the user doing sign language and Watson prints the spelling of each word. We trained Watson to recognize the alphabet of American Sign Language (ASL) by feeding it static pictures of the ASL alphabet. For instance, if someone signs the word Baby, Watson will compile the pictures taken and spell out B-A-B-Y. If time permits, we will also train Watson to translate American Sign Language into other sign languages (such as French Sign Language).”

freshAir – “A bot that notifies you if/when it’s safe to go outside and breathe the air.”

Interview IQ – “Intelligent video interviewing platform that focuses on a constant feedback loop and provides both candidates and hiring manager’s insight on culture fit, overall interview success/failure and key factors in the decision-making process of both parties involved. We believe the result will be a more open and transparent hiring process where talented individuals can learn how to better market themselves and employers will be able to more efficiently identify star candidates that will fit within their company’s culture.”

Callmatters – “The political situation in the U.S. has been volatile (to say the least), lately. One thing that you hear repeatedly is that you should call your representative and tell them that you care about certain issues. However, most of us don’t do it. With call matters, you no longer have an excuse to not contact your representative! The app is intentionally simple. You can call your representative in just a few clicks and tell them about the issues that matter to you. You can also see their record, and set a reminder to check up on the latest news on Capitol Hill.”

AI phone game – “Test you and your friends’ phone game skills with AI phone game. Send a SMS to our app with the phone numbers of the people who you want to play with and play a round of AI Phone Game with your friends. The app scores your abilities at the end of a game.”

Smart Floor – “We’ve built a smart floor for tracking movement within the home as both a security measure and a point of data for tracking high-traffic areas.”

Security Pressure Test on Edison – “The name says it all, it’s a security pressure test on Edison

OnUs – “For people who have trouble sticking with their fitness goals, we’re creating an app for them to workout while being held accountable by us, and their friends.”