Backache? The app uses your webcam to detect bad gestures and says it doesn’t monitor.

Are you lounging at your desk again? There’s an app that does that… Zen uses posture mirroring software to help information workers stop hunching over — sends an alert when it detects you’re not sitting up straight, so you can correct your posture and hopefully A lifetime of back problems can be avoided.

What’s the question? It uses your webcam to check your posture. So, uh, you have to be comfortable with Zen’s software “seeing” through the lens as you work.

Considering how many people routinely cover their webcams with duct tape to make sure — you know, the NSA isn’t looking — that’s quite a ask. So we asked Zen co-founder and CEO Daniel James how the San Francisco-based startup is tackling this privacy concern.

In addition to offering tools directly to consumers (it currently has more than 1,000 users on its subscription service), the startup also sells a version of Zen to employers and has signed up about 30 companies since launching the product in October 2020 (including more than a dozen companies). Given the rocket growth of worker monitoring tools since the pandemic-fueled boom in remote work, there are plenty of reasons to worry about privacy.

For example, can employers who sign up for Zen use the tool, if not to monitor employees sitting at their desks (which can be boring), but to record how long they’re actually sitting in front of a screen — and use those data points to force employees to shorten the desk breaks they may wish to take?

The dystopian use of webcam-based tools isn’t hard to imagine, because this kind of stuff, sadly, isn’t science fiction. Take the AI-powered cameras Amazon rolled out in its Prime delivery vans last year, which it said were used to assess the “safety” of drivers, but critics immediately called it Orwellian surveillance. …..

In short, “AI with eyes” might just feel, uh, creepy in functionality.

Zen said it took a “privacy-centric” approach to building this webcam-based posture correction technology — meaning it took concrete steps to try to reassure users that when the AI ​​was watching them , they won’t be spied on by it or anyone else.

First, its posture correction software is open source (the code is on Github). “We use open source software throughout the app, except for our workouts and educational content, which is custom made by us,” James noted when asked about it.

The AI ​​is also processing data on the device — meaning it doesn’t require an internet connection to function — so he says users can verify for themselves that it’s not uploading to the cloud by testing with Wi-Fi/internet connection disabled Transfer any data.

“The postural correction software function operates offline, without internet, and without recording or storing visual effects,” he emphasized. “Because data, like photos or videos, can’t go to the cloud without an internet connection, which is the only way an employer can spy on an employee, it’s not technically possible for us (Zen) or the employer to record or store any visuals,” he said. And end up spying on people.”

He also confirmed that employers using Zen’s software only received “aggregated” (company-wide) and “anonymous” (no individual names) information on how many employees signed up for the app and how many were using it on a weekly basis.

“Through these two data points, they can see whether the employee is involved with Zen, which is often the deciding factor when they decide to extend their contract with us,” he added.

So — to put it bluntly — Zen’s claim is that neither employers who pay for the software (or Zen itself) can access users’ camera profiles to record or store any visuals.

“The app’s posture correction feature, the only feature that accesses personal cameras, doesn’t run in the cloud, which technically means no one can go into the app and access the data, including visual effects,” James said, adding . “Surprisingly, no employer has ever asked for ‘spy’ type data. They really just want to know if employees are using the solution they’re paying for.”

Even so, privacy-conscious office workers may still dislike the idea of ​​sitting in front of naked cameras all day.

After all, nothing in tech is as blissfully invisible as a webcam with stickers on it.

Posture Check
On that note, James thinks Zen users have figured out their own way of getting a feel for the tool, as he says they tend to use the app to do brief posture checks — say, every day with the feature enabled 30 minutes or 1 hour instead of keeping it on all the time.

The developers took advantage of this cleverly — recommending that users do only one brief check per day to maintain posture awareness.

“We’ve noticed that most of our users don’t leave Zen on all day. Instead, they do a short 30-60 minute pose session the first time they work in the morning, and again around noon or 4pm. It’s related to how most people meditate,” James told TechCrunch. “The short amount of time they spend actively being aware of their thoughts reinforces their passive awareness throughout the day. It also works great for gestures.

“Doing short subsections every day ultimately increases your posture awareness and behavioral change. Also, in user interviews, people often tell us, ‘even when I’m not at the table, Zen seems to be in the back of my head. I find myself at the dining table On Lazy, I started noticing it naturally and moved back to an upright position’. From this anecdotal and analytical data, we decided to recommend people to start using the app in what we call the ‘7 Day Posture Challenge’, In just 30 minutes a day, people have seen amazing results in improved posture awareness and reductions in back and joint pain.”

“We found that when people realize that Zen doesn’t have to be on all day, they’re less worried about any privacy being violated. You can turn it on and off however you want,” he added.

James also said that users often use posture monitoring in conjunction with other apps that require the webcam to be turned on, such as when they’re on a video conference call.

“They can demonstrate gesture awareness and confidence during video calls, and they have their cameras turned on when they’re using Zoom and G Meets, so they don’t worry about any privacy violations,” he noted.

Zen integrates with the user’s general PC workflow — running in the background and reflecting the user’s posture through the stick man icon displayed in the menu bar, allowing the user to keep tabs discreetly without being interrupted by alert messages. Blue and upright are good; curved and red are bad. (James says it never sends distracting/pop-up informational alerts; but users can choose how they want to be alerted from several options.)

How Zen alerts users to bad posture. A larger graphic can be pinned to the screen (with an optional alarm function), or users can rely on watching a discreet stick man in the menu bar (source: Zen)

Posture-correcting AI works against a user-defined baseline — meaning the user needs to demonstrate their upright posture during setup. The app then uses this to build a user-specific model consisting of vectors that record key pose points/indicators (joints, nose, ears, etc.), so the AI ​​can detect pose changes in real-time (i.e. when cameras monitor when enabled) and determine if the person is slouching.

“These pose points are fed into a mathematical model that constantly compares your current pose position to the original baseline pose you set to the ‘upright’ position,” he explained. “In addition, the app applies geometric formulas to the vector formed by your current pose position and your original baseline upright pose position to determine if you’re slouching.”

There is a personal reason for James’s passion for good posture, he was a “very active” college football player in Yale’s NCAA Division I before working at Adobe in San Francisco — “living a typical long Sitting company life, sitting in front of the computer for more than 8 hours a day” – which eventually caused him to suffer from severe low back pain and carpal tunnel.

“Adobe has provided great ergonomic resources like free ergonomic consultations and standing desks, and purchased different devices that claim to help improve posture, but my pain continues to increase,” he said. Details what led to the creation of Zen.

And one more luck: His co-founder Alex Secara — his roommate at the time and now Zen’s CTO — had developed it for himself in college A posture correction software to help with his spine-related ailment (kyphosis), which was exacerbated by a long coding job during a tech internship.

“We ultimately decided to join forces and build Zen with top ergonomics experts and physical therapists,” James added.

Zen disclosed $3.5 million in up-front funding from Y Combinator, Valor Equity Partners, Goodwater Capital, Samsung Next, Softbank and other investors, which it said it will use to invest in expanding its team’s growth and product development — – Plan to integrate (Slack, G Cal, Microsoft Teams) on key workdays, and develop software versions for different devices/platforms (phones, tablets, etc.).

The startup also told us that it is exploring partnerships with larger companies and “further demonstrating the effectiveness of our solution through clinical studies.” (Current enterprise clients include Brex, Alation and Cedar).

Zen also plans to switch the consumer product to a free version — it says it aims to build a model similar to meditation app Calm’s premium paid features.

Expanding sales of physical products (more ergonomic chairs, mice, keyboards, etc.) is also on its roadmap, James said, and he also wants to explore whether it can take advantage of the availability of more high-end headphones, wearables, and mobile phones. There is motion sensor hardware to see if these signals can be repurposed to determine if a person is slouching.

If it can design an AI model to solve this problem, it may eventually be possible for users to get real-time, back-saving posture correction prompts without having to turn on the camera. happiness!