Adobe’s GenStudio content supply chain platform is now generally available, with the ability to publish content directly to social media channels such as Instagram, Snap and TikTok coming soon.
Adobe launched GenStudio for Performance Marketing — as the standalone GenStudio application is now called — in preview at Adobe MAX conference in 2023. At this year’s MAX event it made a slight change in branding: GenStudio now refers to both the GenStudio for Performance Marketing app and the various Adobe applications it integrates with, such as Adobe Experience Manager, Adobe Express, and Workfront.
Adobe has been quick to integrate its Firefly generative AI models across Creative Cloud apps such as Photoshop and Illustrator, enabling designers to increase their output significantly, the company says. (IDC analysts also predict genAI will boost marketing team productivity by 40% in the next five years.)
The aim of GenStudio for Performance Marketing is to help marketers access and use the AI-generated content created within their organization while respecting brand guidelines and legal compliance policies.
“The challenge facing most brands out there is that they have an inefficient content supply chain, where bottlenecks appear in areas like planning, content development and measurement,” said Varun Parmar, general manager for GenStudio at Adobe, in a news briefing. This is where GenStudio for Performance Marketing can help, he said, providing a “seamless way for brands and agencies to deliver on-brand and personalized content that is compliant with brand standards.”
GenStudio for Performance Marketing performs several functions. First, it serves as a content repository where users can access pre-approved assets such as images, logos, and videos for use in the creation of marketing content. This could be anything from display ads to banners and emails. To enable reuse of content across campaigns, GenStudio for Performance Marketing integrates with Adobe Experience Manager Assets, Adobe’s digital asset management app.
Firefly video
Users can also edit and adapt existing assets from the app using the Firefly AI models. This could mean creating variations of email ads tailored to a specific geographic region, for instance.
Those models will soon include new video capabilities, including text-to-video and image-to-video, now available as beta versions.
In GenStudio for Performance Marketing, an AI-powered “brand check” feature can automatically inspect assets before they are used in marketing campaigns, comparing with pre-defined templates and alerting marketing and design teams where content may be out of step with a firm’s brand compliance guidelines. Here, each asset is given a score out of 100, with detailed recommendations for changes: an email headline that’s too lengthy, for example, or innappropriate tone of voice. An integration with Adobe’s Workfront also enables automated “multi-step review workflows,” to provide additional oversight of the approval process
Adobe also plans to let users publish content directly from GenStudio for Performance Marketing to social media channels from the likes of Meta, TikTok and Snap, as well as display ad campaigns with Google’s Campaign Manager 360, Amazon Ads and Microsoft Advertising. This campaign activation feature is “coming soon,” Adobe said, without providing further details. It will also be possible for customers to publish content via their own email and web channels via Adobe Journey Optimizer in future, Adobe said.
Finally, GenStudio for Performance Marketing will provide analytics on the performance of content that’s live on platforms owned by Meta (such as click-through rate, cost per click and spend), with integrations with others such as Microsoft Advertising, Snap and TikTok also available “soon.”
“All companies have to ramp up their genAI knowledge and its impact on brand content/assets,” said Jessica Liu, principal analyst at Forrester.
“Solutions like GenStudio present compelling opportunities for companies to alter their creative development and production process — such as creating more content, accelerating workflows, streamlining workflows, or shifting workforce skillsets.”
Ad customization comes at a customized price
Adobe hasn’t published a list price for GenStudio and GenStudio for Performance Marketing. A company representative said, “As this is enterprise software, there isn’t a one size fits all pricing as it’s based on the customer need/requirement.”
In a potentially alarming development for global cybersecurity, Chinese researchers have unveiled a method using D-Wave’s quantum annealing systems to crack classic encryption, potentially accelerating the timeline for when quantum computers could pose a real threat to widely used cryptographic systems.
In a potentially alarming development for global cybersecurity, Chinese researchers have unveiled a method using D-Wave’s quantum annealing systems to crack classic encryption, potentially accelerating the timeline for when quantum computers could pose a real threat to widely used cryptographic systems.
In a potentially alarming development for global cybersecurity, Chinese researchers have unveiled a method using D-Wave’s quantum annealing systems to crack classic encryption, potentially accelerating the timeline for when quantum computers could pose a real threat to widely used cryptographic systems.
In a potentially alarming development for global cybersecurity, Chinese researchers have unveiled a method using D-Wave’s quantum annealing systems to crack classic encryption, potentially accelerating the timeline for when quantum computers could pose a real threat to widely used cryptographic systems.
Microsoft Excel offers a plethora of tools for representing your data visually. The most basic of these — and arguably the most useful — is the humble chart. But it’s not always easy to know where to begin with charts. We’re here to help.
In this tutorial, we’ll get you started using charts and sparklines in Excel, from understanding the basic chart types to creating and editing charts based on your spreadsheet data.
In this article
What is an Excel chart?
What are sparklines?
Common chart types in Excel
Creating a chart
Editing a chart
Using sparklines
What is an Excel chart?
A chart is a visual representation of the data in an Excel worksheet. Charts allow you to easily see trends, make comparisons, and gain insights that are hard to see from just the raw numbers.
What are sparklines?
Sparklines are tiny charts that are placed within a single cell and used to visually represent trends in your data. While charts typically show an entire data set in one diagram, sparklines show a trend in a row or column, so having multiple sparklines on the same spreadsheet is not uncommon.
Common chart types in Excel
Excel offers a large variety of chart types to choose from. These range from popular general styles such as bar, line, and pie charts to highly specialized styles aimed at particular fields or types of data, such as waterfall charts for financial data. In this story we’ll focus on the most commonly used chart types.
Most of the major chart types have several subtypes — when inserting a bar chart, for example, you can choose from among six subtypes: clustered bar, stacked bar, 100% stacked bar, and 3-D varieties of each. For details about subtypes of the common chart types, see this Microsoft support page.
To get you started using charts, here are the most common types of charts used in Excel and when you’d want to use them:
Column and bar charts: These chart types are very similar, with column charts showing values vertically and bar charts showing them horizontally. Both types are best suited for showing changes in data over time or for quick comparisons. For example, the following simple column chart shows total sales year over year.
A column chart is good for comparing values over time.
Shimon Brathwaite / IDG
Line and area charts: These chart types are best suited for showing changes, particularly small changes, over short or long time periods. Line charts show trends with one or more lines stretching across a grid, whereas area charts fill in the vertical spaces between lines with different colors, highlighting how parts relate to the whole. For this example, let’s look at hypothetical changes in a company’s stock price for 2024 in a line chart.
A simple line chart clearly shows small fluctuations in data values.
Shimon Brathwaite / IDG
Pie and donut charts: These chart types show how individual parts compare to the whole and are best used with data sets where no values are negative, zero, or close to zero. Pie charts can show only one data series; donut charts are similar but arrange the data in concentric rings, allowing them to show more than one data series. In this example, the pie chart shows the age ranges of the company’s customer base.
Pie charts are ideally suited to showing portions of a whole.
Shimon Brathwaite / IDG
XY (scatter) and bubble charts: These charts are best used to show the relationship between two variables. For example, the scatter chart below shows the relationship between age and average earned income. Bubble charts are similar but use variably sized bubbles instead of dots to indicate values.
A scatter chart illustrates how one value relates to another.
Shimon Brathwaite / IDG
These are just a few of the charts that Excel supports. To learn about more chart types that you may need for specialized uses, please see Microsoft’s chart types support page.
Creating a chart
Now that you know the basic chart types, we’ll go over how to get them into your spreadsheet. Copy and paste the following sample data set into a blank Excel worksheet if you want to follow along.
2022
2023
2024
Basketballs
10,000
11,000
12,000
Footballs
8,000
12,000
14,000
Soccer Balls
14,000
12,000
10,000
To create a chart from your data set, first select the whole data set, then choose one of the following options:
Use the Recommended Charts button: The first (and usually best) option for creating a chart is to let Excel suggest which type of chart to use. To use this feature, go to the Insert tab on the Ribbon toolbar and select the Recommended Charts button. This option will examine the data you have highlighted and recommend the best charts to represent it properly. Scroll through the recommendations and choose the chart you want.
Excel’s first recommendation is a clustered column chart that’s perfect for showing the values in the sample data set.
Shimon Brathwaite / IDG
Select your own chart type: If you’d rather choose your own chart type, go to the Insert tab and, to the right of Recommended Charts, select the icon for the chart of your choice — in the example shown below, the pie chart icon. A panel will appear letting you choose the chart subtype you like.
Inserting a pie chart.
Shimon Brathwaite / IDG
Use the Quick Analysis tool (Windows only): Excel for Windows has a handy Quick Analysis tool that you can use to create charts and more. To use this, simply highlight the data that you want to use for the chart, then select the icon that appears at the bottom right corner of the data. On the pane that appears, select Charts and choose from any of the recommended charts that are present.
Tip: If Excel has trouble understanding elements of your data set — for example, failing to recognize column headers as such — try converting your data to table format first. (See our Excel tables tutorial for instructions.) Then select the table and proceed with one of the options above.
(One more way to create a new chart is to start with one of Microsoft’s premade chart templates and customize it to your liking. But a template is meant to provide a framework for you to fill in with your own data, not something you apply to an existing data set.)
Once a chart has been added, you can resize it by selecting any corner and dragging it to enlarge it. To move a chart on the spreadsheet, click and hold the white space next to the chart title, then drag your chart wherever you would like to place it.
Drag any corner to enlarge the chart.
Shimon Brathwaite / IDG
Editing a chart
Once your chart is in place, you’ll likely want to edit it in various ways, such as adding data labels or changing its formatting. We’ll use the column chart generated using the Recommended Chart option in the previous section for demonstration purposes.
Customize the chart title: Excel typically uses the placeholder text “Chart Title” at the top of the chart, so the first thing to do is change that to something more appropriate. Double-click the chart title and type in a new name. Let’s call it “Sales Data.”
Type in a new chart title.
Shimon Brathwaite / IDG
Change, add, or remove a legend: The legend is the portion of the chart that explains what each column relates to. In our example, the legend has three items: 2022, 2023, and 2024. If you want to change these years, simply select the cell entry that corresponds to that item. For example, if you want to change 2022 to 2021, modify cell B1 and change it to 2021. The chart will auto-update. (Before we proceed with the demo, change 2021 back to 2022 again.)
If your chart doesn’t include a legend, you can add one: select the chart, go to the Chart Design tab on the Ribbon toolbar, click the Add Chart Element button, and select Legend from the menu that appears. Next, choose a location for the legend: Right, Top, Left, or Bottom. To remove a legend if you don’t want one, follow the same steps and choose None.
Add data labels: Data labels add numeric values to a chart, rather than relying solely on visualization. To add data labels, select the chart, navigate to the Chart Design tab, and click Add Chart Element > Data Labels > Outside End. In our example, the numeric labels appear at the top of each column.
Adding data labels to the chart.
Shimon Brathwaite / IDG
Format chart elements: Multiple chart elements can be changed to create different visual effects. To change the chart’s overall appearance, select the chart, go to the Chart Design tab in the Ribbon toolbar, and choose from among the different designs shown. For our demonstration, we’ll select the design with the dark background.
Select a new chart design from the row of options along the top.
Shimon Brathwaite / IDG
To change the colors of the columns within your chart, stay on the Chart Design tab, select the Change Colors button, and choose a new color scheme.
Changing the color of the chart’s columns.
Shimon Brathwaite / IDG
Change the chart type: If you want to see what your data would look like in a different type of chart, simply select the chart, go to the Chart Design tab, select the Change Chart Type button toward the right end of the Ribbon bar, and then select the chart type that you want. In this case, let’s use a pie chart.
You can use the Change Chart Type option to see how a different type of chart would show your data.
Shimon Brathwaite
As you can see, a pie chart doesn’t suit this type of data well and doesn’t show each product’s years. Undo this change using the Undo button at the top of the Excel window before you continue.
Click Undo to change the chart back to a column chart.
Shimon Brathwaite / IDG
Swap X and Y axes: Excel also has a built-in button for swapping the X and Y axes in a chart, which gives you a different view of your data. To use this feature, select your chart and, on the Chart Design tab, select the Switch Row/Column button. In our example, the chart now groups the data by year, with a column for Basketballs, Footballs, and Soccer Balls within each year group.
Swapping the X and Y axes in your chart can give you a different perspective on your data.
Shimon Brathwaite / IDG
Note that swapping X and Y axes doesn’t work well with some data sets and might result in a jumbled, hard-to-read chart. If this happens to you when you’re experimenting with charts, simply undo the swap and move on.
Add a trendline: A trendline is simply a line that shows the trend of data in a chart. Starting from our example chart with the X and Y axes swapped (so the columns are grouped by year), select the chart, go to the Chart Design tab, and choose Add Chart Element > Trendline > Linear. On the Add Trendline dialog box that appears, choose Footballs and click OK. A trendline appears on the chart showing the trend in football sales over the three years.
A trendline can highlight a particular trend in your data — in this case, the rise in football sales over time.
Shimon Brathwaite / IDG
Update or filter the data shown in a chart: You can update your chart at any time by modifying the data set from which it is pulled. To illustrate this, go to cell B3 and change the football sales for 2022 to 15,000. You will notice that the chart — including the trendline — automatically updates.
When you change a value in a data set, Excel will automatically update a chart based on it.
Shimon Brathwaite / IDG
If you’ve formatted your data set as a table, you can also filter data from the data set, and these changes will be reflected in the chart you’ve created. The best way to do this is through the use of slicers, which are buttons you can use to easily filter data in Excel. See our Excel slicers tutorial for information about using slicers and charts together.
Add a secondary axis: In some instances, based on the type of data being represented, you may want to have a secondary axis. This can help you highlight how two different types of data series relate to one another, especially if their values are dissimilar.
To see how it works, use the following sample data set to create a column chart using the first recommended chart type (Clustered Column):
Month
Units
Defect(%)
January
500
5
February
400
3
March
450
6
April
375
8
May
250
12
The resulting chart shows both “Units sold” and “Defect(%)” columns for each month, but it’s hard to draw any conclusions from the data.
A traditional clustered column chart doesn’t work well with this data set.
Shimon Brathwaite / IDG
Next, select the chart, go to the Chart Design tab, select Change Chart Type > Combo, and choose the second combo chart option. This changes the chart so that the Defect(%) column is graphed as a line with its own secondary vertical axis on the right. Now we can clearly see the correlation between units sold and defect percentage.
For certain data sets, adding a secondary axis can highlight data correlations.
Shimon Brathwaite / IDG
Using sparklines
Sometimes you don’t need a full chart but simply want to highlight a small trend within the data. That’s where sparklines come in. They’re in-cell visualizations that can show the trends within individual rows or columns within Excel. You can think of them as mini charts for small subsets of your data.
To add a sparkline, select a blank cell where you want to add it — typically at the end of a row or the bottom of a column. In this case, select cell E2. Then select Insert > Sparklines > Line. The Create Sparklines dialog box appears asking you to select a data range for the sparkline. If it’s not already selected, select cells B2 to D2 and click OK.
Selecting a data range for the first sparkline.
Shimon Brathwaite / IDG
The sparkline will be added to cell E2. To add sparklines to the end of all the rows, select the green square at the bottom right corner of the first cell, drag it down over all the cells where you want sparklines to appear, and release your cursor.
Sparklines are great for showing at-a-glance trends for subsets of your data.
Shimon Brathwaite / IDG
Now we can see at a glance the trends in sales for basketballs, footballs, and soccer balls.
To edit your sparklines, click on any sparkline and then go to the Sparkline tab in the Ribbon toolbar. From here, you can change the color of your sparklines by selecting another option shown in the toolbar or by clicking the Sparkline Color button. You can also change the sparkline type — for example, from line to column.
It’s easy to change the look of sparklines.
Shimon Brathwaite / IDG
Sparklines give you a great option for building data visualizations into your data sets rather than creating a standalone chart to summarize or explain your findings.
With charts and sparklines in your Excel toolkit, you’re well on your way to highlighting the most important parts of your data clearly and intuitively.
AMD’s big Advancing AI event in San Francisco on Thursday underlined how quickly the microprocessor industry has pivoted to artificial intelligence (AI) as its main sales pitch.
The company offered three hardware announcements across its processor line-up, each appealing to different parts of the AI market.
The first was the new Instinct MI325X AI accelerator chip, a datacenter-oriented GPU which ups performance on every metric compared to last year’s MI300. The company also showed off its fifth-generation EPYC processors for the enterprise cloud and datacenter sector. And it unveiled the new Ryzen AI PRO 300 series, a family of processors for mobile PCs aimed at enterprise buyers.
Mobile chips have traditionally been low power (and lower performance versions) of their desktop equivalents, but with the focus on AI, that distinction is fast disappearing.
AI requires more raw power, which is now showing up in the specifications for new chips. For example, the Ryzen AI PRO 300 series offers three processors, starting with the Ryzen AI 9 HX PRO 375; it features 12 Zen 5 cores/24 threads, a clock speed that can be boosted to 5.1GHz, and integrated Radeon 890M graphics.
It also features a neural processing unit (NPU) that delivers up to 55 tera operations per second (TOPS), making it the most powerful desktop AI chip of its kind on the market.
Just below that in the line-up is the Ryzen AI 9 HX PRO 370, an identical chip with slightly less NPU performance — up to 50 TOPS. And the entry-level chip is the Ryzen AI 7 PRO 360 with 8 cores/16 threads, a 5GHz clock speed, Radeon 880M graphics, and the same 50 TOPS NPU performance.
NPUs are on the new frontline of desktop competition because they make possible features such as accelerating Microsoft’s Copilot + and AI-intensive tasks such as real-time language translation.
They also pit AMD against traditional rival Intel, which has had NPUs of its own in its Core Ultra CPUs since 2023. Today, NPU AI acceleration is premium priced, but there are signs the technology is likely to jump quickly to more mainstream chips.
“With Ryzen AI, we’ve actually enabled hundreds of different AI functions,” said AMD CEO Lisa Su near the end of a two-hour Advancing AI presentation. “Our latest software stack makes it really easy for developers to optimize thousands of pre-trained models for Ryzen.
“Our Ryzen AI Pro 300 series resets the bar for what a business PC can do,” she said.
Though there are patches affecting Windows, SQL Server, Microsoft Excel and Visual Studio, only the Windows updates require a “Patch Now” schedule — and they’ll need a significant amount of testing because they cover a lot of features: networking, kernel and core GDI components and Microsoft Hyper-V. Printing should be a core focus for enterprise testing and the SQL Server updates will require a focus on internally developed applications.
The team at Readiness has crafted this infographic outlining the risks associated with each of the October updates. A rundown of recent Patch Tuesday releases is available here.
Known issues
There were a few reported issues for the September update that have now been addressed, including:
These are relatively minor concerns compared to dealing with recent problems deploying Windows 11 24H2. Covering both compatibility and security challenges, these include:
TheSafe Exam browser may fail to load. Version 3.7 of this application is currently “hard-blocked” by Microsoft until further notice. This means Microsoft has updated the list of applications that are currently not allowed to run on the target platform.
Fingerprint sensors and readers may not function as expected. According to Microsoft, a firmware update should resolve the issue.
Compatibility issues with specific sound cards (Intel Smart Sound) could cause them to stop working properly.
These problems are likely to be resolved with application and firmware updates rather than Microsoft patches and primarily affect users upgrading to Windows 11 24H2. That said, Microsoft has advised there are problems with the “first build” or out-of-boxinstallation of this latest Microsoft release. We suggest that enterprises wait until the next release before serious testing and deployment.
Major revisions
This month, Microsoft published the following major revisions:
CVE-2024-38163: Windows Update Stack Elevation of Privilege Vulnerability. This is a low-level administrator (WinRe) vulnerability that has neither been publicly exploited nor disclosed. This is a documentation update; no further action is required.
CVE-2024-38016: Microsoft Office Visio Remote Code Execution Vulnerability. This “remote code” security issue actually requires local access to succeed. It has not been reported as exploited in the wild and Microsoft has provided an official fix. This is a documentation update only; no further action needed.
Testing Guidance
Each month, Readiness analyzes the latest updates and provides detailed, actionable testing guidance based on a large application portfolio and the patches’ potential impact on the Windows platforms and app installations.
We’ve grouped the critical updates and required testing into separate product and functional areas including:
Microsoft SQL Server
With two updates this month, desktop (or client) testing will be required for data-driven applications. We recommend that the following SQL-related tests be included for October:
Validate SQL Commands and stored procedures.
Ensure data “Refresh” operations perform correctly with Microsoft Active Data (ADOX) objects. These are difficult operations to debug due to the generally large number of inter-connected objects (databases and systems) and the business criticality of these systems. Start early on this effort.
Test queries that accept large numbers of parameters. SQL parameterboundary testing is probably a good idea.
Windows
While the primary testing scenario for this update is really to test printing, there is a lot to check. Microsoft has made significant changes to broad areas in networking, low-level changes to the Kernel and graphics handler (GDI), and updates to core features including Microsoft Hyper-V. A feature-by-feature testing regime should include:
Networking: Test large file transfers (include IPv6) over remote desktop connections, VPNs and varied network conditions. Web browsing tests should include multiple simultaneous connections — and messaging applications such as Microsoft Teams should be included in this cycle.
Security: Ensure that (internal) code still performs cryptographic functions accurately using RSA keys. Authentication should work correctly between both Microsoft and Linux systems. A validation of Kerberos client authentication will also be required.
Remote Desktop: updates to Microsoft Routing and Remote Access Server (RRAS) server will require remote access administrative action testing. Remote desktop licensing will require functionality testing. And the remote desktop related APIs MprConfigFilterSetInfo and MprInfoBlockRemove have been updated, so internally developed systems that connect with RRAS will require an authentication test.
Windows Error Logs: Due to a change in the Windows Common Logging File System (CLFS) a quick test of resultant container files is required.
Again, the primary focus should be on testing printing. Rather than a simple (does it actually print) test, more complex print-related checks are required, including:
Validating text rendering and formatting for entire documents;
Starting, stopping and disabling printer queues;
Printing across a “matrix” of 32- and 64-bit platforms that includes variations of both desktop and server environments. The main challenges will be found with 32-bit applications on 64-bit platforms (Adobe Reader, we’re looking at you).
Install and uninstall third-party software management software on both platforms.
Windows lifecycle and enforcement updates
This section includes important changes to servicing, significant feature deprecations and security-related enforcements across the Windows desktop and server platforms.
Windows 11 Enterprise Version 21H2 Microsoft servicing support ended on Oct. 8, 2024.
Mitigations and workarounds
Microsoft published the following mitigations applicable to this Patch Tuesday.
CVE-2024-43609: Microsoft Office Spoofing Vulnerability. Microsoft has published additional documentation on setting Group Policy Objects (GPOs) referencing the Restrict Outgoing NTLM traffic to remote servers policy that will reduce the scope of this security issue through improved connection request auditing and reporting.
CVE-2024-38124: Windows Netlogon Elevation of Privilege Vulnerability. While not offering specific settings or security configurations, Microsoft does offer advice on how to reduce the impact of this vulnerability with best practice guidance on server naming conventions, name change reporting/auditing and employing multi-factor authentication.
Each month, we break down the update cycle into product families (as defined by Microsoft) with the following basic groupings:
Browsers (Microsoft IE and Edge)
Microsoft Windows (both desktop and server)
Microsoft Office
Microsoft Exchange Server
Microsoft Development platforms (ASP.NET Core, .NET Core and Chakra Core)
Adobe (if you get this far)
Browsers
Microsoft released just three updates for the Chromium browser project specific to Microsoft Edge:
The Chromium project has provided a veryhandy dashboard for its latest releases and testing status. Add these browser updates to your standard release schedule.
Windows
Microsoft released one patch with a critical rating and 92 patches rated important. This month, the following key Windows features have been updated:
Microsoft published six updates (all rated important) for the Office platform. These updates do not include any preview pane or reported zero-click vulnerabilities and only affect Excel and SharePoint. Add these to your standard Office update schedule.
Microsoft SQL (nee Exchange) Server
There were no updates for Microsoft Exchange Server. However, Microsoft released two updates to Microsoft SQL Server product group (CVE-2024-43481 and CVE-2024-43612); add them to your standard server update schedule.
Microsoft development platforms
Microsoft released a single update rated critical (CVE-2024-43488) to Visual Studio and eight further updates (all rated important) to the Microsoft .NET platform. None of these security issues have been reported as exploited or publicly disclosed, so add them to your standard developer release schedule.
Adobe Reader (and other third-party updates)
Microsoft did not publish any Adobe Reader related updates. That said, there are critical updates for both Reader and Acrobat that deserve attention. Microsoft included an update for another third-party application (CURL) that addresses a free memory buffer overflow vulnerability (CVE-2024-6197) — just like Reader used to do). The assigning CNA for this issue is named as HackerOne, which we find endearing.
Apple didn’t cast much light on visionOS at WWDC this year, and it hasn’t received much attention since. But don’t mistake this something for nothing. Two recent events indicate there’s a lot going on behind the scenes.
The first is the release of Submerged, the first movie filmed in Immersive Video written and directed by Academy Award-winning filmmaker Edward Berger (All Quiet on the Western Front).
The second is new research from Apple’s Machine Learning teams that shows how to create accurate depth-of-field data from single-lens cameras using conventional computers.
An immersive movie about immersion
Submerged is a claustrophobic, adrenaline-fuelled, 17-minute story set on a sinking ship — in this case a war-damaged submarine — capturing the crew as they fight to stay alive. The movie is made for Vision Pro devices, and reviewers already claim it delivers a sense of immediacy and intimacy they’ve never experienced before.
All of this is interesting, but how can this kind of experience be delivered in an even more powerful way? How can Apple’s technologies support an even more immersive user experience?
That’s what I think Apple is working on based on the second event to have emerged in the last few days: the introduction of an AI-based model Apple calls Depth Pro.
AI provides depth
What this does is powerful. The AI can basically map the depth of a 2D image. The technology behind it seems similar to what you’d expect if you were building an autonomous vehicle, given such vehicles must be able to accurately determine depth using images of nearby objects in real time.
Apple’s researchers seem to have developed this tech so it will run accurately on an iPhone. They claim that apps using the Depth Pro model can produce accurate depth maps based on images captured by a single lens camera in just 0.3 seconds when run on a computer running a standard GPU.
The team says the tech could have big implications for robots, real-time mapping, and improved camera or video effects. You can read the company’s research paper on these features here, or its post concerning Depth Pro on the company’s machine learning website.
Information is power
Being able to take it to the movies suggests Apple now has a technology that can automatically figure out depth from 2D images. Of course, a movie is just a sequence of 2D images, which means the company has tech to figure out spatial positioning based on what you see on screen.
You can already see this to some extent in that visionOS can turn existing photos into spatial images, adding depth to create a stereoscopic effect. It also makes sense to use that tech to generate 3D environments from 2D images.
What next? In August 2023, Apple researchers published a paper explaining FineRecon, which showed how 3D scenes constructed from posed images using AI can be made more accurate and deliver scenes that offer more fidelity. That research couples well with earlier information concerning a project to deliver enhanced 3D indoor scene understanding.
Movies you can walk through
Combine all these ingredients and, in theory, the breakthrough Apple might achieve could involve the creation of tech that can both understand images, and also add to them. After all, if you know that object A is in one position and object B in another, you can more easily deliver the illusion of walking between or even behind those objects to a Vision Pro user.
Generative AI (genAI) solutions already exist that can create video or image “fakes,” but to what extent can the computer exploit its knowledge of depth of field to generate 3D experiences in which you can literally walk behind the objects you see? And how could those technologies be applied to the viewing experience of watching Apple’s Submerged movie?
Even as it is, the experience of being in a sinking submarine is immersive in both senses of the word — but being able to find your own viewpoint within that action in high fidelity would realize every video gamer’s dreams. It would certainly sell a few movies.
Arranging the scenery
It’s important not to jump too far forward. Building technologies to achieve these things is going to be much more challenging than simply pontificating on the possibilities in prose, but there are other potential visionOS implications to the application of accurate depth-of-field data based on 2D images. I’m particularly thinking about use in emergency response, medicine, remote drone control — even space exploration, and all from a single-lens camera, making the tech lightweight and highly portable.
In other words, along with new frontiers for creative expression, there are viable business opportunities about to be unlocked by Apple’s home-grown reality distortion machine. Will we see some of them emerge with visionOS 3.0 at next year’s WWDC? Is it then we’ll really see how Apple Intelligence can work miracles with Spatial Reality?
