OpenAI makes 4 major announcements for developers at DevDay 2024

Together with

Welcome Humans🤖,

Here is what we have Today:

• Voice to Vision: OpenAI's 4 Game-Changing Dev Tools

• AI stethoscopes & baby monitors: Future of health

• New Job Opportunities

Trending Today

• Meta announced Orion, “the world’s most advanced AR glasses ever made” that comes with eye, hand and neural tracking.

• Microsoft has announced a slew of AI upgrades coming to its Copilot assistant for Windows PCs, including new vision and voice capabilities, personalization enhancements, a re-release of the controversial Recall feature, and more.

• Meta released the open-source code and developer suite for its Segment Anything Model (SAM) 2.1, an upgraded version of its image and video segmentation tool.

• Lapsi Health, a medtech startup has secured FDA approval for a new digital stethoscope that uses AI to detect chronic heart and lung conditions. Next up: A wearable device for pregnant mothers that can be used to track their baby’s movements and alert them to emergencies in advance.

• OpenAI rival, Anthropic, has hired one of OpenAI’s lesser-known co-founders—Durk Kingma—although no one knows exactly what he’ll be doing, or which team he’ll be joining or leading.

• Pinterest revealed a new AI feature that can replace plain product image backgrounds with realistic lifestyle imagery to help advertisers improve their ads.

Together With Writer

Writer RAG tool: build production-ready RAG apps in minutes

RAG in just a few lines of code? We’ve launched a predefined RAG tool on our developer platform, making it easy to bring your data into a Knowledge Graph and interact with it with AI. With a single API call, writer LLMs will intelligently call the RAG tool to chat with your data.

Integrated into Writer’s full-stack platform, it eliminates the need for complex vendor RAG setups, making it quick to build scalable, highly accurate AI workflows just by passing a graph ID of your data as a parameter to your RAG tool.

Learn more about our production ready RAG tooling here.

OPENAI

⚡OPENAI SUPERCHARGES DEVELOPERS

OpenAI launched a public beta of its "Realtime API" at DevDay 2024, enabling developers to create AI-powered voice conversations. Also, announcing four major API features, focusing on accessibility, efficiency, and cost-effectiveness for developers.

What can the new API do?

• Offers six custom voices for speech-to-speech interactions

• Enables real-time verbal conversations for tasks like trip planning

• Integrates with maps and calling APIs

• Provides low-latency, AI-generated voice responses

What are the other features?

1. Model Distillation

• Simplifies fine-tuning of smaller models

• Uses outputs from larger models

• Makes training more accessible

2. Prompt Caching

• Reduces costs by nearly 50%

• Speeds up responses by up to 80%

• Reuses recent input tokens in API calls

3. Vision Fine-Tuning

• Allows training with both images and text

• Optimizes image recognition and analysis

• Enables new visual AI applications.

What about internal changes?

• CTO Mira Murati and Chief Research Officer Bob McGrew have departed

• CPO Kevin Weil emphasizes continued innovation

• Company serves over 3 million developers globally

What's missing?

• No updates on the GPT Store

• No news about the Sora video generation model

• No information on the anticipated o1 model

How will this impact developers?

• Lower costs for API usage

• Faster response times

• More accessible model training

• New possibilities for voice and vision applications

What's next?

• Developers to implement AI disclosure for voice calls

• Potential for new applications using real-time voice interactions

• Possible acceleration in AI app development

• Future announcements expected for missing features

While less flashy than previous events, these updates could significantly impact how developers build and scale AI applications. This development marks a significant step in AI voice technology, though some anticipated features remain unreleased as OpenAI navigates leadership changes and market competition.

Recommended Reading
If WE had to recommend other newsletters

AI Tool Report is an Newsletter, we enjoy reading daily. AI Tool Report delivers top techniques on how AI can transform your business filled with practical tips, real-world examples.

A Smart Bear is for people who love thinking about strategy, startups, product, marketing, decision-making, and founder psychology.

Job Opportunities

• Waymo - Software Engineer, Motion Planning and Dynamics - Apply

• Databricks - Sr Staff Software Engineer (Backend) - Apply

• Meta - Growth Marketing Manager, B2B, AI - Apply

• UiPath - People Operations Specialist - Apply

• Meta - Art Manager, Wearables - Apply

AI ART

Side-by-side shot of two loaf variations: one traditional chickpea bread loaf topped with rosemary and olive oil, and another vegan version with added flax seeds and nutritional yeast.

Ideas? Comments? Complaints?

We read your emails, comments and poll replies daily.

Get the most important AI, tech, and science news in a free daily email.
New Here? Subscribe!
Sponsorship Slots Open for October and Reach over 800+ active readers. (Now 40% off) 🤯

Until next time, Stay Informed!