First announced at Google’s I/O developer conference this May, Google Photos’ AI-powered search feature, “Ask Photos,” is rolling out to users starting Thursday. The feature, which allows users to ask the AI to find photos using more complex queries, will initially be available in “early access” to select customers in the U.S. before expanding to a wider user base.
Powered by Google’s Gemini AI model, Ask Photos lets users search their photos using natural language queries that leverage the AI’s understanding of their photo’s content and other metadata. Previously, Google Photos users could search for particular people, places or things in their photos, but the AI upgrade will allow them to ask a larger variety of questions, including those that require a deeper understanding of the photos.
Image Credits: Google
For example, as Google suggested during I/O, you could ask for the “best photo from each of the National Parks I visited.” The AI uses a variety of signals to determine what’s the “best” of a given set, including things like lighting, blurriness, and lack of background distortion, among other things. It would then combine that with its understanding of the geolocation of the photos to find those taken at National Parks.
Google said the feature could be used for more than just photo retrieval alone; users would also be able to ask questions to get helpful answers. For instance, a parent could ask Google Photos what themes they had used for their child’s last four birthdays. The AI would be able to analyze party photos and determine if a theme was involved, like “mermaid,” “princess,” “superhero” or anything else. It could then tell the parent when those themes were last used.
Image Credits: Google
More practical questions could be those that help you recall a specific event, like “what did we order last time at this restaurant” — presuming you like to photograph your meals. Or you could ask “where did we camp last time” at a specific destination, like Yosemite, the company suggests. You could also use the feature to help you put together photos for an album or to summarize all the things you did on a trip.
The AI understands the context of your photo gallery, including the important people in your life, your hobbies, your favorite foods, and other relevant details and memories.
To access Ask Photos, select U.S. users will be able to find the feature within Google Labs, as it’s still in the experimental phase. The company says the feature’s development is guided by its AI Principles, and the private data in Photos will never be used for ad targeting. However, Google employees may review users’ queries to help improve the AI over time. The AI’s answers will not be reviewed by humans, unless the user reaches out for support, to provide feedback, or to report abuse or harm.
Interested users can sign up on the waitlist for early access to Ask Photos.
Navin Chaddha, the leader and managing partner of 55-year-old VC firm Mayfield Fund, has a penchant for approaching venture investing in a way that deviates slightly from other established firms.
When Mayfield raised a $955 million fund last year, Chaddha told TechCrunch the firm didn’t need a multibillion-dollar fund because “copying somebody else is strategy for disaster, strategy for failure.”
The firm is again trying to do something that it sees as unique. On Wednesday, Mayfield said it is launching AI Garage, a $100 million initiative for ideation-stage founders interested in building “AI teammate” companies.
AI Garage wants to distinguish itself from accelerators such as YC or pre-seed programs like Sequoia’s Arc or Greylock’s Edge by modeling the effort on its entrepreneur-in-residence (EIR) experience. For the past 40 years, Mayfield Fund has hired one or two EIRs each year and helped them turn a raw concept into a new, fundable company.
With the new program, Mayfield plans to scale and formalize its EIR program by welcoming up to five aspiring founders into its office every six months.
Just like with EIRs, AI Garage participants won’t receive capital on day one, but Mayfield will allocate a minimum of $1 million and as much as $5 million as soon as the business plan is hatched with the help of a firm’s partners and other support staff, including marketing, talent and business development team.
As for why Mayfield decided to quintuple and formalize its EIR program, the answer comes down to Chaddha’s interest in getting early access to AI application startups, specifically in the area he calls “AI teammates.”
Image Credits: Mayfield
“They haven’t even started the company. We will help them start it,” Chaddha said.
According to Chaddha, AI teammates differ from copilots and agents because they are more than simple assistants, which can answer questions or perform actions autonomously, like booking meetings or offering refunds. “Teammates collaborate with humans on complex tasks to achieve a shared goal,” he said. “AI teammates are digital companions that elevate humans to superhumans. They are going to take us into a new era of collaborative intelligence.”
Although the terms “copilot,” “agent” and “teammate” can be used interchangeably, labeling an AI app as a teammate can be seen as a clever marketing tactic because it just seems more human-friendly.
“We believe there are endless opportunities for AI teammates to collaborate with humans and to shape the future of our workplace by having AI work together with humans in many areas, including product and engineering, data, sales and marketing, customer service, IT and security, finance, HR, legal and many administrative functions,” Chaddha said.
Mayfield has already invested in nearly a dozen AI teammate companies, including DevRev (customer service support AI), Docket (an AI sales engineer) and NeuBird (site reliability AI engineer.)
In NeuBird’s case, human site reliability engineers tell its AI to detect any site outages, and then triage and troubleshoot problems. If the AI discovers it can’t fix the problem, it calls human engineers for help. “That’s an example of a teammate,” Chaddha said.
Prevention is better than cure, as the saying goes. Today, a splashy startup that has taken that concept to heart — literally and figuratively — is expanding.
Neko Health was co-founded by Spotify’s Daniel Ek and Hjalmar Nilsonne with a mission to improve preventative healthcare with annual full body scans and AI-powered insights that can detect conditions like heart disease and skin cancer. Now, the Swedish company is launching in London, its first city outside of its home market.
For £299 and an hour of your time, the company runs a series of scans and tests that it says creates “millions” of datapoints that help determine your state of health across a range of cardiovascular, metabolic, and other conditions. An in-person meeting with a doctor to discuss the data is included in the session. Those interested in getting in line for a visit can sign up here.
The company is coming to the U.K. on the heels of a strong start in Stockholm, where it has opened two clinics in its first year of operation. To date, it has scanned 2,707 people, and while 78.5% of them were found to have no health issues, results for some 14.1% of visitors raised issues that required further medical treatment. Some 1%, it said, were identified to have “severe” cardiovascular, metabolic, or cancerous conditions. Of that group, none were aware of their conditions prior to visiting Neko, the company said.
Neko is Japanese for “cat” and Nilsonne tells me that it’s a reference to cats having nine lives, or more specifically a lot of lucky breaks. “That’s how we would hope that our system feels for our users,” he said.
A visit to the Neko clinic
Despite the participation of a major tech figure like Ek, the company’s ethos is actually very understated. As a result, it sometimes gets in its own way.
When I visited the London clinic to go through the testing process to write this article, it took me a few trips up and down a London street to realise that I had walked straight past the Neko clinic, which was marked only by its logo — an N sliced and shifted through its waistline — at the top of a glass doorway. The rest of the operation is below ground level, making the street-level lobby look more like a discreet entry to a fancy apartment block rather than a health center.
Image Credits: Neko Health (opens in a new window) under a CC BY 2.0 (opens in a new window) license.
Once you enter the building and descend the stairs, you enter an equally sparse lobby with pared-down furniture, muted colours and lighting, and low-volume, soothing electronic music piped throughout. The concept is somewhere between elegant day spa and futuristic medical clinic.
Once there, you’re funnelled through their system. An assistant takes a few notes about you before leading you into a room where you strip down to your underpants, a robe, and slippers. A second assistant then measures your height and weight before moving you to a cylindrical chamber that feels like a cross between a tanning booth and a security scanner at an airport.
The chamber itself is fitted with 70 sensors, including dozens of volumetric and other cameras, to produce a composite picture of you with 50 million data points. From there, you move to an examination bed, where you have another two scans, measurements of your eye pressure and blood pressure, and a grip test. Finally, the the clinician draws a blood sample.
The blood sample is the only invasive test that Neko carries out, and altogether the experience takes under an hour. Soon after, you change and are taken to a separate consultation room where a doctor goes through your data.
Neko’s focus is on risks and proactive prevention rather than existing conditions. Data points are grouped and presented along the lines of different potential problems you might encounter related to them — among them skin cancer (my moles were counted and it seems that I have many more than the average person), high blood pressure, and metabolic syndrome (a group of conditions that can lead to illnesses such as heart disease, stroke, and diabetes).
If the data reveals problems, you’re referred for more diagnostics and channeled to doctors for further examinations.
Unlikely beginnings
Nilsonne tells me that Ek first approached him to chat about healthcare over a Twitter direct message.
The year was 2018, and Nilsonne’s company at the time — an AI-powered,smart home energy monitoring startup — was running out of money and winding down after failing to find product-market fit with the big energy companies it expected to be its customers.
Image Credits: Neko Health (opens in a new window) under a CC BY 2.0 (opens in a new window) license.
“Then out of nowhere, I get a direct message on Twitter from Daniel.” Ek had just taken audio streaming platform Spotify through an IPO and was a newly minted billionaire looking for another thorny problem to solve, something that would have “a positive contribution to the world,” in Nilsonne’s words. He’d gotten it into his head that he wanted to reinvent healthcare, just as he’d reinvented recorded music consumption with Spotify.
“So he reached out to me. We didn’t know each other, but he was aware of what I had been doing, working with hardware and software and AI.”
Ek offered to fund a startup with Nilsonne at the head on the spot, but Nilsonne at first rejected him. Neither of them have a medical background. But Nilsonne does come from a family of doctors so it’s in his consciousness, and Ek was obsessed, so the two kept talking, and they could see something starting to take shape.
Wearables like Fitbits and Apple Watches, Nilsonne said, have led to an “incredible explosion of abundance of high-quality health information.” Combining that with AI and the ability to draw “sensible conclusions” from all this data, “it was very clear that we could create a different healthcare system.” Thus began five years of building prototypes for the physical clinics and their component hardware, as well as the software to analyse the data.
Neko itself, plus its co-founders Nilsonne and Ek, come from an unlikely location. Sweden is home to one of the world’s most famous socialised medicine programs. You might assume that the existence of free, and good, healthcare would make a paid service like Neko a hard-sell to consumers. But so far at least, the opposite has been true.
When the company last year announced a $60 million funding round, it had a waiting list of “thousands” lined up to get scans. Now that list is up to 22,000.
Yes, some of that might be down to Neko being the latest project from one of Sweden’s most famous living entrepreneurs (who is also one of Neko’s major investors by way of his investment vehicle, Prima Materia). But Nilsonne believes Neko is addressing an important gap that will give the startup staying power.
The majority of healthcare services, including those in Sweden and the U.K., are focused on treating conditions rather than preventative care. But because our populations are aging, the number of conditions that need treatment are rapidly increasing, and that’s putting the whole system under pressure “in an unsustainable way,” Nilsonne said, explaining that 75% of healthcare costs go towards chronic diseases.
“Our hypothesis is that prevention and early detection could to be the things that would solve a lot of these problems. Most chronic conditions are fully preventable, or you can delay them by decades if you have effective early interventions. But of course, in our system, you basically show up when you already have the disease, and then it’s too late to reverse them, and there are a lot of costs associated with it.” (It is far from being the only startup focusing on preventative healthcare technology. Zoi in France and Germany’s Aware are two in Europe;
In the U.K., the NHS does offer a Health Check that covers many of the same areas that Neko does, although as Nilsonne points out, it’s less frequent.
“The NHS health check is available only once every five years to individuals aged 44 to 74,” he points out. “It offers a much narrower range of tests and does not include a consultation with a doctor to discuss the results. We know from our data that chronic diseases are increasingly appearing in individuals under 44” — which also happens to be the average age of a Neko customer — “Therefore, it’s crucial to adopt a proactive approach to health to identify potential issues early, take corrective action, and monitor progress over time.”
Andreessen Horowitz’s partner Joshua Lu knows that, in the video game industry, you can never get too comfortable. When he was head of product at Zynga, he experienced the height of mobile games, working on hits like Words with Friends; then as a vice president at Blizzard Entertainment, he helped produce tentpole hits like Diablo Immortal. And then, as a director of product management at Meta, he learned to see games in new dimensions while working on the VR game, Horizon Worlds.
“I had to forget what I thought were universal truths and learn a whole new set of ways to do things,” Lu told TechCrunch.
Now Lu wants a front row seat to where video games are heading. After joining the firm as an investor in 2022, Lu helped launch the firm’s Speedrun accelerator, which invests $750,000 apiece into about 40 gaming startups twice a year. Now on the firm’s third cohort — with the applications for the fourth cohort now open — Lu said he’s seen how AI and new distribution platforms are changing the industry.
Half of the accelerator’s current batch are AI companies, doing everything from creating AI-crafted stories to using AI for 3D avatars. “The last game that I worked on at Blizzard took six years and a $250 million budget to ship,” he said, referring to Diablo Immortal. “But wouldn’t it be so great if that kind of quality of game could be done with a 10th of the budget and a 10th of the people?”
We might quibble with how great it is for AI to kill high-paying developer jobs at the largest game companies. But if AI also helps more startups form and be qualitatively competitive, that’s a compelling thought.
Lu says he’s seen firsthand how companies are getting creative, citing Clementine, a startup that went through Speedrun. The company “released a demo where you had to solve a mystery by talking to AI and making sure that they didn’t find out that you were a human,” he said. That may be a terrifying premise, or a tongue-in-cheek one, depending on how existential a threat you think AI could become.
Lu also mentioned Echo Chunk, a company that raised $1.4 million in a round led by Speedrun. Echo Chunk went viral for its game Echo Chess that uses AI to instantly generate an endless number of levels. “These are all fairly early explorations,” he said. “But we’re excited in general about novel types of game design interactions and game dynamics that can be unlocked because of AI.”
Lu is also advocating for startups to build games atop Discord. Earlier this year, Discord made it so developers can create apps for people to use within the chatting platform. Lu said that, over the course of his career, he’s seen the places for people to discover games dwindle; for example, no one finds games through social media feeds anymore, like many did with Farmville. “Where can we find the next platform where truly social games can be created and distributed?” Lu said.
Several companies entered the accelerator building within Discord. Lu said several more pivoted to building in Discord over the course of the 12-weeks. “There are more games being made than ever, and it’s hard for developers to stand out,” he said. He hopes building on Discord will help “people to find pieces of content that they would really like playing.”
Jam & Tea Studios is the latest gaming startup implementing generative AI to transform the way players interact with non-playable characters (NPCs) in video games.
Traditionally, video game NPCs are directed by predetermined scripts, which can feel repetitive, unrealistic and boring. It also may restrict the number of potential experiences for players. However, when generative AI is involved, players can engage in casual conversation and interact with NPCs how they want to (within reason).
Founded by gaming veterans from Riot Games, Wizards of the Coast and Magic: The Gathering, the company announced on Friday its first game that will leverage generative AI tools to help with gameplay mechanics, content generation, dialogue and item generation.
Jam & Tea’s debut game, Retail Mage, is a roleplaying game that allows players to take on the role of a wizard working as a salesperson at a magical furniture store. The main goal of the game is to earn five-star reviews by helping customers. But it’s really up to the players to decide if they actually want to work or cause chaos. With AI NPCs as customers and human players being able to say and do almost whatever they want, the possible outcomes should vary widely.
In Retail Mage, players are approached by customers who each have their own requests. Instead of selecting from preset phrases, players can type in the text generator how they’d like to respond. The player can ask the AI to “say something charming,” and it will offer four different dialogue options.
Image Credits: Jam & Tea
Jam & Tea is among several companies competing in the AI-powered NPC space, alongside Artificial Agency, Inworld and Nvidia. Ubisoft’s AI-powered “Ghostwriter” tool writes NPC dialogue for some of its games.
The new game also comes at a time when there’s concern among creatives about the potential challenges posed by the prevalence of generative AI. Last month, SAG-AFTRA — the union comprised of voice actors and other talent — initiated a strike against major game publishers over AI concerns.
However, Jam & Tea claims it’s taking a balanced approach to the inclusion of AI, and wants to protect artists, writers and other creatives working in game design.
“Our philosophy is that we believe creatives are going to be only more essential as we move forward in using this technology and in bringing new experiences to players,” co-founder and chief creative officer M. Yichao, who was the former narrative designer for Guild Wars 2, League of Legends and other titles, told TechCrunch.
“AI will generate all this dialogue, and you can talk to characters endlessly… but it’s going to take the creative eye and lens to really add meaning to that and to craft that into an experience that matters into something with impact, depth and emotion that carries through stories. That’s going to become more important than ever,” Yichao added.
He explained that creatives are heavily involved throughout the development process, including when it comes to crafting NPCs, giving them motivation, interests and backstory, as well as providing example lines to help the AI mimic the tone and generate lines in real-time.
Limitationsof AI NPCs
Despite its advantages, generative AI in NPCs has its limitations. One major concern is the issue of AI unpredictability, when the behavior of an NPC becomes excessively erratic, resulting in a frustrating experience for the player. AI can also hallucinate answers, so there’s a possibility that the NPC could say something that’s wrong or doesn’t exist in the world.
Continuously improving the AI engine will help mitigate unpredictable NPCs, Yichao believes. Players can also rate the characters’ responses, which provides data to help improve the characters’ behavior. Plus, Jam & Tea claims to have put guardrails in place to prevent inappropriate conversations.
Players are still encouraged to be creative, allowing for inventive and spontaneous interactions to occur. For example, instead of helping a customer, players can choose to engage in activities instead, like playing hide and seek — a real scenario that occurred during playtesting.
“Our lead engineer was playtesting one night and went up to the NPCs and just said, ‘I’m bored.’ And the NPC responded by saying, ‘Well, why don’t we play a game? Let’s play hide and seek.’ And so the other NPCs heard and said, ‘Oh, we’re playing too,’” shared co-founder and CTO Aaron Farr. The NPCs proceeded to follow the rules of the game, with one seeker walking throughout the store to find all the hiders.
“None of that was programmed; all of that was emergent behavior. That is part of the delight of when we have what a player wants to do combined with its experience to modify the experience in real-time,” added Farr, a former engineering leader at Riot Games and Singularity 6.
The company has been experimenting with various large language models (LLMs) throughout the testing phase, including OpenAI, Google’s Gemma, Mistral AI and Meta’s Llama, and other open models. It’s currently uncertain which LLM it will ultimately use in the final version of the game, but is fine-tuning the model to train it on how to give better responses that are more “in character.”
Generate items out of thin air
Jam & Tea’s AI engine goes beyond dialogue generation. Players can also interact with any object in the game and state their intentions with that object, such as picking it up or dismantling it for parts. They can even create items from scratch. Depending on what they want to do, the game interprets that intention and determines if they’re successful or not.
In a demo shown to TechCrunch, Yichao interacted with an NPC named Noreen, who asked for an antelope-shaped plush. He then typed a command into an action box and retrieved a pillow resembling an antelope from a crate. The game recognized his action as successful and added the item to his inventory.
Because the item didn’t previously exist in the game, players won’t physically see an antelope-shaped plush appear. All that happens in the game is the item shows up in the player’s inventory as a default image of a pillow. If the player wants to perform an action, like sitting in a chair, a notification appears on the screen indicating that the action was performed.
“One of the things that’s really exciting about this technology is it allows for open-ended creative expression. Like, I can take a piece of meat and say, what if I put it in the bowl and I make a delicious fish stew? We might not have a fish stew [image], but one of the things that I’m working with our artists on is coming up with a creative ability to represent that item in a way that’s satisfying in the world and allows the player’s imagination to fill in some of those blanks, and gives players maximum creative freedom to make things that are unexpected,” Yichao said.
AI technology won’t be used for 2D or 3D asset generation. Real artists will create the images.
Image Credits: Jam & Tea
Retail Mage is a relatively basic game compared to others. At launch, the company promises to provide a more advanced product than the test version we saw during the demo.
Jam & Tea states that the game is primarily intended to demonstrate the application of the technology as it continues to experiment. Beyond Retail Mage, the company is also developing another game — currently referred to as “Project Emily” internally — which will showcase their broader ambitions, featuring more environments and a sophisticated storyline.
The startup’s scrappy team of eight has a lot of work ahead to reach the level of bigger gaming companies. However, taking action now while there is momentum allows the company to adapt and grow as AI models advance.
Jam & Tea raised $3.15 million in seed funding from London Venture Partners with participation from Sisu Game Ventures and 1Up Ventures. It plans to raise another round later this year.
As for the business model, Jam & Tea will charge $15 to buy the game and offer extra game packs that players can purchase separately. It’ll launch on PCs initially, but the company aims to enable cross-platform functionality within the next few years.
Retail Mage is slated to be released to the public later this fall.
Image Credits: Wikimedia Commons (opens in a new window)
Given everything you’ve already heard about AI, you may not be surprised to learn that Google is among other outfits beginning to use sound signals to predict early signs of disease.
How? According to Bloomberg, Google has trained its foundation AI model with 300 million pieces of audio that included coughs, sniffles, and labored breathing, to identify, for example, someone battling tuberculosis.
Now, it has teamed up with an Indian company – Salcit Technologies, a respiratory healthcare AI startup — to tuck that tech into smartphones where it can potentially help high-risk populations in geographies with poor access to healthcare.
It’s not Google’s first foray into trying to digitize human senses. Its venture arm has also backed at least one startup that’s using AI in an attempt to sniff out disease, literally.
Image Credits: Jakub Porzycki/NurPhoto / Getty Images
TikTok announced on Monday that it’s introducing generative AI avatars of creators and stock actors for branded content and ads on its platform. The company is also launching an “AI Dubbing” tool for creators and brands to expand the reach of their ads and branded content.
The new “Custom Avatars” are designed to represent a creator or a brand spokesperson. Creators can choose to scale their likeness to create multilingual avatars to expand their global reach and brand collaborations, TikTok says. Brands can create avatars with their spokesperson or a creator they have partnered with to localize their global campaigns.
There are also new “Stock Avatars,” which are pre-built avatars created using paid actors who are licensed for commercial use. The idea behind these avatars is to give businesses a way to add a human touch to their content. TikTok says the avatars are created with actors from a range of backgrounds, nationalities and languages.
Image Credits: TikTok
The launch of the new tools might be concerning for some. Hollywood actors went on strike last year over several different concerns about AI, including the fear that their likenesses would be used to generate AI replicas. With TikTok’s new AI avatars, creators have control over whether they want their likeness used. Creators can also determine their rates, licensing and who can use their avatar.
As for the new AI Dubbing tool, it will allow creators and brands to translate their content into 10 languages, including English, Japanese, Korean and Spanish. It automatically detects the language in a video, and then transcribes and translates the content to produce a dubbed video in the preferred language. TikTok says the new feature lets creators and brands communicate with global audiences.
The launch comes as TikTok recently revealed that 61% of users have made a purchase either directly on TikTok or after seeing an ad.
The new features will be available as part of “TikTok Symphony,” the company’s suite of ads solutions powered by generative AI, which was launched in May. The suite includes tools to help marketers write scripts, produce videos and enhance current assets.
TikTok is building out its ads business despite facing a potential ban in the U.S. if its parent company, ByteDance, fails to sell the app within a year.
With the advent of generative AI, AI applications are transforming and reshaping various industries and changing how people work. Software development is no exception.
San Francisco- and Tokyo-based startup Autify has built an autonomous AI agent for software quality assurance to help software engineers complete code quickly, streamline workflows and increase productivity. In other words, it’s built an AI QA engineer for software engineers.
The startup announced Monday that it had closed a $13 million Series B round and rolled out its beta version of Zenes, an AI agent for software quality assurance tailored for customers in the U.S. The Series B funding brings its total raised to $30 million, which consists of $24 million in equity and $6 million in debt financing.
Global Capital Partners and LG Technology Ventures co-led the latest round, along with existing backers World Innovation Lab (WiL), Salesforce Ventures, Archetype Venture and Uncorrelated Ventures.
In 2016, two former software engineers, Ryo Chikazawa (CEO) and Sam Yamashita, co-founded Autify after they experienced firsthand the pain point of software testing lengths. To address this issue, Autify has built a platform called Autify NoCode, which enables developers and QA teams to improve efficiency and enhance the software quality engineering process, ultimately saving time and resources.
NoCode caters to users in Japan and South Korea who heavily rely on manual testing. Currently in beta, Autify’s newly launched Zenes is specifically designed for U.S.-based customers.
Zenes “generates test cases by analyzing product requirement documents, writes automated test codes and maintains the test codes automatically,” according to the company. The company claims that Zenes significantly reduces the time to create test cases by 55%, a feature that sets it apart in the market.
“Companies that have been heavily investing in automation by writing code don’t need to adapt to no-code/[low-code] because they can code. However, they are still suffering from the lack of resources,” Chikazawa said, adding that the companies expect generative AI to make their work more productive. “With the launch of Zenes, now we can capture an even earlier stage of software quality assurance which is designing and creating test cases and we will be able to provide a comprehensive end-to-end solution for the entire QA process.”
This is how Zenes works: Users can upload their product requirement documents in various formats, including .html, .pdf, .docx, and .md. The agent then generates a set of outlined test cases covering the product specs. Users can edit these test cases in order to improve the accuracy of the following steps. After any edits have been made, Zenes generates automated test codes.
“We believe that AI is not here to replace humans. It’s here to enhance human capability so that we can be more creative. The software development process and its quality assurance will be redefined with generative AI,” Chikazawa said.
The startup will use the new capital to implement more AI capabilities to support customers’ software QA, Chikazawa told TechCrunch. In addition, the startup has partnered with LG CNS, an IT service unit of LG Corp, to expand into the Korean market, along with its core markets, the U.S. and Japan.
Autify did not disclose the number of customers but said it offers its services in 16 countries now, up from two countries in 2021, when it raised its Series A. Autify has B2C and B2B customers, including DeNA, NEC, NTT Smart Connect, Yahoo and ZoZo. Its staff has also more than tripled from 30 in 2021 to 100.
DeepMind, Google’s AI research lab, says it’s developing AI tech to generate soundtracks for videos.
In a post on its official blog, DeepMind says that it sees the tech, V2A (short for “video-to-audio”), as an essential piece of the AI-generated media puzzle. While plenty of orgs, including DeepMind, have developed video-generating AI models, these models can’t create sound effects to sync with the videos that they generate.
“Video generation models are advancing at an incredible pace, but many current systems can only generate silent output,” DeepMind writes. “V2A technology [could] become a promising approach for bringing generated movies to life.”
DeepMind’s V2A tech takes the description of a soundtrack (e.g. “jellyfish pulsating under water, marine life, ocean”) paired with a video to create music, sound effects and even dialogue that matches the characters and tone of the video, watermarked by DeepMind’s deepfakes-combating SynthID technology. The AI model powering V2A, a diffusion model, was trained on a combination of sounds and dialogue transcripts as well as video clips, DeepMind says.
“By training on video, audio and the additional annotations, our technology learns to associate specific audio events with various visual scenes, while responding to the information provided in the annotations or transcripts,” according to DeepMind.
Mum’s the word on whether any of the training data was copyrighted — and whether the data’s creators were informed of DeepMind’s work. We’ve reached out to DeepMind for clarification and will update this post if we hear back.
AI-powered sound-generating tools aren’t novel. Startup Stability AI released one just last week, and ElevenLabs launched one in May. Nor are models to create video sound effects. A Microsoft project can generate talking and singing videos from a still image, and platforms like Pika and GenreX have trained models to take a video and make a best guess at what music or effects are appropriate in a given scene.
But DeepMind claims that its V2A tech is unique in that it can understand the raw pixels from a video and sync generated sounds with the video automatically, optionally sans description.
V2A isn’t perfect, and DeepMind acknowledges this. Because the underlying model wasn’t trained on a lot of videos with artifacts or distortions, it doesn’t create particularly high-quality audio for these. And in general, the generated audio isn’t super convincing; my colleague Natasha Lomas described it as “a smorgasbord of stereotypical sounds,” and I can’t say I disagree.
For those reasons, and to prevent misuse, DeepMind says it won’t release the tech to the public anytime soon, if ever.
“To make sure our V2A technology can have a positive impact on the creative community, we’re gathering diverse perspectives and insights from leading creators and filmmakers, and using this valuable feedback to inform our ongoing research and development,” DeepMind writes. “Before we consider opening access to it to the wider public, our V2A technology will undergo rigorous safety assessments and testing.”
DeepMind pitches its V2A technology as an especially useful tool for archivists and folks working with historical footage. But generative AI along these lines also threatens to upend the film and TV industry. It’ll take some seriously strong labor protections to ensure that generative media tools don’t eliminate jobs — or, as the case may be, entire professions.
Climate change is raising the risk, rate and cost of sewage failures. Floods are becoming more common, leading to backups that frequently overwhelm wastewater treatment systems. Exacerbating the issue, America’s infrastructure is woefully outdated; the EPA estimates that nearly $700 billion in investments is needed to simply maintain existing wastewater, stormwater and other clean water pipelines over the next 20 years.
Matthew Rosenthal and Billy Gilmartin, both of whom hail from the wastewater treatment industry, saw an opportunity to help solve the problem with tech — in a small way, at least. Five years ago, the pair co-founded SewerAI, which taps AI to automate the types of data capture and defect tagging that make up a sewer inspection.
“Most infrastructure was built post-WW2 and is reaching the end of its useful life, leading to more frequent failures and increased costs,” Rosenthal told TechCrunch. “SewerAI revolutionizes underground infrastructure inspection and management with its AI-driven software-as-a-service platform.”
SewerAI began as Rosenthal’s side project; he’d started taking online courses on AI after co-launching two wastewater analysis and services firms. While experimenting with AI models to predict sewer defects in inspection videos, Rosenthal recruited the help of Gilmartin, who was working at a sewer inspection company at the time.
Image Credits: SewerAI
Today, SewerAI — whose customers span municipalities, utilities and private contractors — sells cloud-based, AI-powered subscription products designed to streamline field inspections and data management of sewer infrastructure.
One of those products, Pioneer, allows field inspectors to upload inspection data to the cloud and tag issues — data that project managers can then use to plan fixes to pipes. Another tool, AutoCode, automatically tags inspections of pipes and manholes, creating 3D models of infrastructure from videos captured on a GoPro or other camera.
“Legacy incumbents offer on-premise or on-truck software that has seen very little innovation in the last 20 years,” Rosenthal said. “SewerAI’s technology increases top and bottom lines by enabling more inspections per day at a lower cost.”
SewerAI isn’t alone in the nascent market for AI-assisted pipe inspection. The company’s rivals include Subterra, which maps, analyzes and forecasts problems with pipelines; ClearObject, which offers software that analyzes footage from pipe inspections for damage; and Pallon, which develops algorithms to spot potential problems inside sewers from still images.
Image Credits: SewerAI
What sets SewerAI apart, Rosenthal claims, is the quality of its data — specifically the quality of its model training data. Rosenthal says that SewerAI has footage of inspections of 135 million feet of pipes from municipalities and independent contractors. While just a fraction of the 6.8 billion feet of sewer pipes in the U.S., it’s a large enough dataset to train a competitive defect-detecting AI, Rosenthal says.
“Our products streamline field inspections and data management, enabling clients to proactively manage infrastructure instead of reacting to emergencies,” Rosenthal said.
SewerAI’s sales pitch won over investors like Innovius Capital, who along with others poured $15 million into SewerAI’s most recent fundraising round, bringing SewerAI’s total raised to $25 million. The cash will be put toward go-to-market expansion, AI model training, hiring and expanding SewerAI’s product portfolio beyond inspection tools.
“SewerAI is continuing to grow, and we’re seeing an acceleration in demand for our platform as we enable people to do more with existing budgets, which has resulted in us closing our first seven-figure contracts,” Rosenthal said.
