Sony just wrapped up its 50th anniversary celebration, and it’s a chance to show off all the amazing advances in technology that Sony is currently working on behind closed doors.
Normally, the media isn’t allowed to peek into this world of research and development, but this year Sony decided to throw some exciting glimpses at the projects and concepts that will make their way into products in the future.
• Sony is showing us the future of AI, but where does that leave the creators? (Opens in a new tab)
Using artificial intelligence to perfect digital photos and videos
Using artificial intelligence and deep learning models, Sony hopes to be able to reduce digital defects in images and audio including noise and distortion. This is Sony’s take on technology that we’re starting to see more often with its latest camera phone (Opens in a new tab)s.
Using algorithms trained by processing thousands of images, videos and audios, Sony’s Deep Generative Modeling (DGM) can significantly reduce digital imperfections in an image by identifying what the image is and filling in missing information using machine learning. This technology is already in use to improve noise reduction in Sony’s premium range of noise canceling headphones.
Apple and Google have already made great strides in this area. Apple has its own Deep Fusion technology (Opens in a new tab) It pulls out detail from shadows and highlights without adding noise or artifacts. The latest Pixel 7 from Google (Opens in a new tab) The phones come with a bunch of new AI features including its very good night scene (Opens in a new tab) and reduce opacity (Opens in a new tab) techniques.
As the world’s largest sensor manufacturer, it’s great to see Sony bring this technology to anyone who uses its sensors, so we should see plenty of smartphones capable of these image correction tricks in the near future.
New time-of-flight sensors can open up new opportunities for subject tracking
Sony has made a new time-of-flight (ToF) sensor and software suite for developers who are using their imaging sensors for AR (augmented reality) purposes. ToF sensors are an exciting technology that uses a laser system to track movements in real time.
In Sony’s examples, it was using the system to track the model’s hands in exceptional detail as they moved. The tracking can detect incredibly subtle movements and can also ensure that movements are kept smooth. Also, the tracking is completely unaffected by color. The new ToF AR suite can also be used with artificial intelligence and machine learning to make predictions and estimates about finger movements.
This also has exciting potential for camera tracking and focus systems in the future, with the iPhone (Opens in a new tab) Already pushing hard with ToF features with its LiDAR sensor it can only be a matter of time before we start seeing a lot of this type of system make its way into camera phones and other mirrorless cameras (Opens in a new tab) hardware.
Change the way we watch sports with Hawkvision
Many tennis fans will be familiar with Hawkeye, which has been used for years to make phone calls. It was also recently deployed at the 2022 FIFA World Cup in the latest iteration of the controversial VAR system.
Sony showed off the huge improvements made to the Hawkeye system. The most impressive aspect is the amazing Skeletrak system, which uses cameras all around the field to track players down to the movements of their skeletons. Not only is it useful for making sporting decisions like offside, but the technology’s potential is also huge. The Hawkvision system can use data collected, along with artificial intelligence and machine learning from player stats, tactics, and historical data, to make predictions about player movements.
Now, this is where it gets really interesting. Using Hawkvision also makes it possible to create a 3D model of the entire field and players, so that the game can be followed in virtual or augmented reality. In theory, allowing football fans to choose their own camera angles, even from pitch level. A system that could fundamentally change the way we consume sport.
Create realistic cityscape photos without leaving home
Using terrain, satellite, aerial imagery, sun positioning, and real-time weather reports, Sony’s Mapray technology allows the user to create a realistic view of a realistic place. Using all of this public data ensures that the 3D image provided is as accurate as possible.
Useful for many commercial purposes such as town planning or architecture, this will also have an impact on the entertainment industry with films and photos no longer needing to be shot on location if ideal real-world locations can be conjured instead. This will also have huge benefits for those who are creating content for the metaverse’s rapid expansion. Urban and landscape photographers may now be able to photograph remote cities around the world without having to leave their homes.
Technology has shown that my Sony is still very digital, and reminds me of Microsoft Flight Simulator, but as graphics resolutions change and AI creates more realistic interpretations, we’ll start to see these cities and places like never before.
Change the way you create with real-time audience emotional feedback with VX
We’re used to creating work and then hoping for the best reaction on release, but with Sony’s viewer experience (VX) AI learning system, creators can get detailed emotional feedback from users during production.
Using a system of video cameras that monitor and interpret facial expressions, heart rate sensors, and audio cues such as audience gasps or applause, the VX system can create a real-time graph of the emotional response your content inspires in its audience.
The ultimate goal of this technology is to allow creators time to rework their projects in response to honest audience feedback in an effort to make the final version of the project as successful as possible. While we’ve seen quiz shows for decades, this is a real attempt to use artificial intelligence to dictate how content is shaped and shaped.
You may also like The best Sony lenses (Opens in a new tab). For more information on the topic of artificial intelligence, check out: What is an AI camera? How artificial intelligence is changing photography (Opens in a new tab)