The VR headset maker on Tuesday announced the preview of its Teleport app, which it said will lower the barrier for 3D content creation — a time-consuming and costly process that typically involves high-endequipment and know-how. “One thing holding back VR and 3D applications is just how hard is to create content,” said Patrick Wyatt, chief product officer at Varjo.
He described the Teleport app as “a self-serve way that anyone with a smartphone can start creating their own 3D scenes,” allowing them to share their surroundings with others.
To create a 3D scene, users scan a physical space with their smartphone camera (an iPhone Pro 12 is the minimum requirement for Teleport) — a process that takes several minutes. It’s possible to film indoor or outdoor scenes (anything up to the size of a small town square will work), though more dynamic environments with crowds of people or lots cars could result in blurred footage.
The footage is uploaded to Varjo’s cloud servers to build a high-resolution 3D scene. When accessed via a VR headset, users can then move around the virtual space and view a reproduction of the environment that was recorded.
Given Varjo’s focus on enterprise mixed reality and VR, Wyatt said Teleport can be used for training, planning, and remote assistance. But he sees Teleport as “foundational tech” that could have broad applications. “We’re not too prescriptive on use cases,” he said. “We want to see all the cool things people will do with it.”
While VR environments are often created with computer graphics, photorealism is preferred for certain enterprise purposes. “Much of what businesses want and need has to be as close to the real thing as possible to use those assets for engineering, sales and marketing purposes,” said Anshel Sag, principal analyst at Moor Insights and Strategy. “Without photorealism, it becomes a lot less valuable and powerful.”
Until recently, photorealistic 3D content has been produced either with expensive Lidar scanners that can cost several thousand dollars, or photogrammetry techniques that are more accessible, but still require specialist skills. In both cases, there are limitations on quality, too, said Wyatt.
A key advantage of Teleport is the use of a machine learning technique called Gaussian splatting. This enables full 3D scenes to be produced from a set of photos, with more realistic lighting, textures, and reflections – ideal for immersive applications such as Teleport, said Wyatt.
Gaussian splatting simplifies the creation of photorealistic 3D environments, though the technology has its own limitations, said Sag. “The biggest challenge for creating 3D content has primarily been the cost and time it takes to generate the assets,” said Sag. “Gaussian splatting is a way to take some shortcuts in the creation of content to make it cheaper and faster with minimal tradeoffs in terms of quality.
“That said, it isn’t without its problems, as the Gaussian splats don’t always come out right or need very specific capture techniques to work right.”
Varjo isn’t the only company to use Gaussian splatting for 3D content creation. Others include Luma and Polycam. Wyatt said Teleport differs in its focus on the creation of 3D environments rather than smaller objects, as well as a need for a higher image resolution so that content can be viewed effectively on a VR headsets.
Apple’s artificial/machine/generative AI research team seems to be opening up as it explores new frontiers in this research, publishing more than 20 new Core ML models for on-device AI through the popular AI community site Hugging Face.
It’s a real change in the company’s customary rectitude in being open about what it’s doing, and it seems likely the move comes in response to demands from its research teams to be a bit more transparent.
Cutting-edge AI capabilities
As first reported by VentureBeat, Apple has released dozens of Core ML models, complementing them with extensive datasets. The company seems to be posting new collections at a rapid clip — the latest item appeared in the collection within the last 24 hours. The collection is extensive and highlights two of the main aims of Apple’s teams: to build models that will eventually run on the device, and to ensure these also preserve user privacy.
Some of the AI functions promised by all this code includs tools for image classification, depth segmentation, text analysis, translation, and more.
What, who, why?
They cover a wide range of applications, including FastViT for image classification, DepthAnything for monocular depth estimation, and DETR for semantic segmentation.
The models are not intended for mass market use and are aimed at developers, who can download them, convert them to CoreML format, and then deploy them in their own code. The process for this was explained at WWDC 2024 in a presentation that details how the models can be deployed once converted. It is also worth noting CoreML is much, much faster in iOS 18, as Apple said.
The models available on Hugging Face are also ready to run at the edge. In addition to better privacy and security, on-device LLM models should also run far more swiftly than cloud-based code.
Apple is also working with Hugging Face on other AI-related tasks, including via the MLX Community. All in all, the company seems to have become more visibly open to open-source contributions as it seeks to build Apple Intelligence.
Not the first time Apple’s been open
Except, that’s not exactly the case. Apple is an active player in open-source development, and while this isn’t always fully understood, a cursory glance through company history shows support for the FreeBSD project, a GitHub repository that offers up source code for operating systems, developer tools and more. It also plays an active part across multiple standards bodies, such as Bluetooth SIG.
In other words, some degree of openness already does exist, though it seems to have opened up more for AI.
There’s a reason for this, of course. AI researchers like to collaborate as they explore these new frontiers, and it’s thought Apple’s customary corporate secrecy might have frustrated attempts to put its own work in artificial intelligence on the fast track. This certainly seems to have changed in the last year, as multiple research notes and AI tools have emerged from the company. This latest batch then is completely in keeping with Apple’s new approach, at least, its new tactics related to this part of tech.
Apple is, therefore, learning from the wider industry.
…And the industry is learning from it
Apple’s stance on privacy leads the industry, and as the potential pitfalls of AI systems become more widely understood it seems probable that more companies will follow its lead.
That means an eventual multitude of small models capable of being run on edge devices to perform a variety of tasks. While the capabilities of such models will be limited by a ceiling comprised of processor speed, computational power, and on-device memory bandwidth, Apple’s approach also includes strategic use of highly secured private cloud services, itself a signal to others in the space to follow its example – particularly as increasingly authoritarian and ill-conceived legislation threatens to undermine the security of networked intelligence itself.
Remember former Google CEO Eric Schmidt? He now makes flying AI robots that target and kill autonomously. (Really!)
His robots are in high demand for one simple reason: GPS jamming.
I’ll explain more about Schmidt’s robots below. But first, it’s time to catch up on the rising trend of GPS, cell phone and other signal jamming, which is triggering a global arms race between jamming and anti-jamming technologies.
The FCC crackdown of 2012
All jamming devices in the United States were banned 90 years ago — long before jamming devices even existed. The Communications Act of 1934 explicitly prohibited deliberate interference with radio communications.
Both cell phone and GPS jamming works by “flooding the zone” with white noise in the same frequencies as phone and GPS receivers, basically a denial-of-service attack on the associated range of radio frequencies. But it was the rise in e-commerce that fueled an industry of online jammer sales. In 2012, a bus passenger in Philadelphia wanted some peace and quiet, so he used a cell phone jammer to jam all the phones on the bus. Later that year, the FCC took legal action against 20 online retailers in 12 states for illegally selling jamming devices.
Despite the crackdown, the illegal use of jammers continued. In 2013, RNM Manufacturing in Houston, TX used a jammer to block employees from using their phones at work and was fined $29,250. Not to be out-done by Houston, a Dallas company in 2022 called Ravi’s Import Warehouse also tried to jam employee calls and was also fined by the FCC, this time for $22,000.
Jammers are still available on the black market, which have led to calls for global enforcement of jamming bans. Signal jamming of every kind is illegal in the United States, which is why it might seem surprising to Americans to learn that thousands of commercial aircraft in Europe are put at risk every day by GPS jammers.
The European jamming crisis
The current dramatic rise in GPS jamming is almost certainly done by the Russian military to protect its bases and assets from Ukrainian drone attacks. More than 46,000 aircraft GPS jamming incidents have been reported over the Baltic Sea, Kaliningrad, the Black Sea, the Caspian Sea and the Eastern Mediterranean since August 2021. New incidents are reported every day.
(The website GPSJAM tracks and displays GPS interference in Europe and the Middle East.)
Major airlines like Ryanair (more than 2,300 flights), Wizz Air (nearly 1,400 flights), British Airways (82 flights) and easyJet (4 flights) have been affected by jamming. The GPS jamming has forced some flight cancellations or diversions. Finnair had to temporarily suspend flights to Tartu, Estonia. And a British Royal Air Force plane carrying the UK defense secretary experienced GPS jamming near Kaliningrad in March 2023.
The Ukraine/Russia conflict is a proving ground and laboratory for all kinds of both military and malicious cyberattack technologies.
Specifically, the conflict is the world’s first large-scale drone war. The Ukraine side alone reportedly loses more than 10,000 drones a month, and the country itself has produced more than 1 million drones since the start of the war; it’s also received an unknown number from abroad, including familiar consumer and business drones like the DJI Mavic 2 Zoom, DJI Mavic 2 Enterprise, Autel EVO II Pro, the Bayraktar TB2 and others.
Both sides are using huge numbers of drones for surveillance, reconnaissance, espionage, explosives delivery, hacking, malware delivery, counter-hacking and signal jamming. And while the Ukraine side leads in the creative use of drones, the Russian side is more advanced in drone GPS and signal jamming innovations.
Nearly every effective drone and counter-drone action pioneered and tested in the Ukraine-Russia conflict will almost certainly be used against business and other targets in the years to come. Based on what’s happening in the war, cybersecurity professionals should be aware of the three main areas drones will be increasingly used by malicious actors:
1. Bypassing physical security: Drones can fly over fences, down air ducts and land on roofs to observe security protocols and plan physical attacks using high-quality cameras.
2. Network sniffing and spoofing: Drones equipped with modifiable computers can mimic Wi-Fi networks to steal sensitive information.
3. Denial-of-Service attacks: Drones can perform de-authentication attacks and jam communications.
Another easy prediction is that businesses will be challenged by malicious drone use, given the illegality of jamming in the US.
The military industrial complex gets to work
As Western GPS-guided munitions are increasingly defeated by Russian jamming, the Pentagon is scrambling to innovate in countering the jamming threat. (This is somewhat ironic, given that the GPS system, the mobile cellular system and, in fact, the internet itself were all created by or founded upon Pentagon research programs.)
One approach is to blow up the jammers. The US Air Force awarded a contract valued at around $23.5 million to Scientific Applications and Research Associates to enable guided bombs to home in on — and destroy — jamming equipment.
The Air Force Research Lab is conducting research on using regular smartphones for real-time detection of jamming and spoofing. And while blowing up jamming devices is a short-term, immediate solution, the longer-term solution is to enable drones to work autonomously, without needing to phone home or be controlled remotely.
One fascinating project is the Pentagon’s Rapid Experimental Missionized Autonomy (REMA) program. The project is developing plug-ins or adaptors that can be fitted to ordinary commercial drones that would enable them to carry out their missions autonomously after being jammed. Contracts for the drone-autonomy adapter interface have been already awarded to companies like Anduril and RTX for the hardware and Leidos, Northrop Grumman and SoarTech for the software.
Eric Schmidt’s flying killer robots
White Stork is a secretive startup founded by former Google CEO Eric Schmidt. The company is building small, low-cost ($400) drones that use AI to target and fly into those targets, thus blowing them up with attached bombs. The drones don’t rely on remote control or GPS navigation, but instead use cameras and AI for navigation and targeting. And because they’re low cost, they can be manufactured and deployed on a massive scale.
Schmidt has been actively involved in supporting Ukraine’s war efforts, and travels to Ukraine frequently to meet with Ukrainian generals about using drones in combat. White Stork drones will soon enter the conflict, if they haven’t already.
The future of jamming and counter-jamming
The future of warfare, as well as industrial espionage, terrorism and cyberattacks in general will involve drones in increasing numbers. History tells us that everything the Pentagon builds and buys for the good guys eventually ends up in the hands of the bad guys. That means we’ll likely need not only jamming, but also defensive technologies to counter weaponized drones that don’t rely on radio signals, but instead use AI for autonomous targeting and attacking. Drones are cheap. AI is free. The autonomous drones are coming. We need defenses that are legal to use.
The Olympics this summer will be our first test run. The terrorist group ISIS has circulated detailed manuals on adapting commercially available drones to carry explosives. The idea is to get the how-to information into the hands of “lone wolf” terrorists operating autonomously. The group has also explicitly called on its followers in Europe to launch drone attacks on Paris landmarks like the Eiffel Tower during this year’s summer Olympics.
France has established an anti-drone coordination center at a military base near Paris in light of the threat. And it’s planning to use antiquated technologies like special guns called SkyWall Patrol that shoot nets designed to capture drones mid-flight, and even laser beam devices. That might be sufficient for the low-tech drones they face today, but the AI drones of tomorrow will require more advanced defenses.
While American businesses, enterprises, and law enforcement remain mostly oblivious to the coming threat from drone-based attacks, Europe is proving to be a laboratory for what’s possible there now, and what’s coming to the United States in the future.
Adobe Acrobat users can now generate and edit images in PDFs via text prompts with the addition of Adobe’s Firefly AI image model.
The update gives Acrobat and Reader users access to some of the same image creation and editing capabilities available in Adobe design apps such as Photoshop, without the need to switch tools. This means it’s possible to create new images from scratch using Adobe’s latest Firefly Image 3 model — which Adobe claims will enable greater realism and more control over the images produced compared to previous models — or manipulate existing content with Generative Fill and Remove Background tools.
The Firefly image feature, announced Monday, is available for Adobe Acrobat Standard and Pro customers; they will receive 250 “generative credits” to use with the Edit Image and Generate Image features each month at no additional cost through January 2025 an Adobe spokesperson said. The price after that date point has yet to be determined.
“Adobe has done workers a service by building technologies like the generative capabilities of Adobe Firefly and working to integrate them across a range of their applications and tools,” said Matt Arcaro, IDC research director for computer vision and AI. By building Firefly into familiar apps such as Acrobat, the level of friction involved in accessing a new technology is reduced, he said.
Adobe has also added new features to the AI Assistant for Acrobat and Reader apps announced in February.
The generative AI (genAI) assistant can now summarize information from multiple documents — including PDF, Word, PowerPoint, and other file formats — at the same time. That lets users identify common trends and generate an overview summary with citations that point to the source of information, Adobe said. Users can then direct the AI assistant to format the content for a specific purpose, such as turning it into an email that can be sent to colleagues.
The ability to query multiple documents adds value to Adobe’s “document chat” tool, said Arcaro. “This creates a significant speed up for gathering key insights, but also to put together a more comprehensive perspective across multiple similar or different documents,” he said.
The Acrobat AI assistant can also be used to summarize the transcription of a Microsoft Teams meeting or webinar, or upload a “VTT” transcript file generated in another meeting app.
The Acrobat AI Assistant is available for free until June 28, after which it will cost an additional $4.99 each month for individual Acrobat Standard and Pro subscribers under Adobe’s “early access pricing.” This pricing is available up to Sept. 4.
Adobe didn’t respond to a request for pricing details after this date.
If you have an Apple ID, you can use iCloud with Windows, which means you can access your email, videos, photos, calendars, bookmarks, files and other iCloud data from your PC – useful when you’re at your desk or traveling without an Apple device, or if you are inside Boot Camp on your Mac.
On the surface, the most straightforward approach to using iCloud with Windows systems might appear to be accessing your iCloud account from your favorite web browser.
The app
That’s great up to a point, but to unlock the power of sync across all the devices you have logged into your iCloud account, including your Windows systems, it is far better to download and install Apple’s iCloud for Windows software on your PC. The software is available for free download directly from Microsoft’s app store.
Once installed and set up, you’ll be able to use iCloud to get to all your photos, documents and bookmarks on your Windows PC to update automatically across all your Apple devices, and vice versa, which is what you get when you use iCloud for Windows.
Borrowed from One Drive
The most recently published version of iCloud for Windows uses Microsoft’s Cloud Files API, (One Drive’s Files On Demand feature) which makes the solution more robust and faster.
This means new assets will swiftly sync across all your devices, “enabling users to be more productive offline on mobile devices and quickly share files on iOS,” according to Microsoft Engineering General Manager Giorgio Sardo. In part, this is because the system is smart enough to just download a “breadcrumb” (around 1k in size) of each of your files, downloading the full item on request.
How does iCloud Drive work?
If you already use iCloud Drive, you’ll be familiar with what it does. If not, then the easiest way to understand the feature is as online storage for all the files and projects you are working on. Anything you save into any folder of the iCloud Drive will sync across all your Apple ID devices (it’s very like Dropbox in this). You will also be able to access items saved to iCloud Drive on your Mac, iPhone, or iPad within this folder on your PC using File Explorer. Fully searchable, the service also lets you share your files with others.
How do I access iCloud using a browser?
At those times when it makes sense to use the browser to access your iCloud data — such as when travelling and using a borrowed machine. It’s easy: just navigate to iCloud.com using your browser and log in using your Apple ID and password.
In a neat touch, any active Reminders you might have will appear at the top of the iCloud main page when you log in. You’ll also find an icon-based set of links to take you to your iCloud apps and services, including your iCloud Drive.
Mail
Contacts
Calendar
Photos
iCloud Drive
Notes
Reminders
Find iPhone
Account Settings
You’ll also find fully functional online versions of Pages, Numbers and Keynote, which you can use to create documents compatible with both Mac and PC. You may also find News Publisher, if you have enabled that app.
Accessing iCloud using a browser is especially useful on borrowed machines, during emergencies, while visiting Internet cafes or even using a machine you don’t own that you happen to have access to. Just remember to log out.
Of course, the beauty of iCloud is its ability to sync across all your devices, but to sync from a Windows system you need to install iCloud for Windows.
How to set up iCloud for Windows
Apple’s iCloud for Windows software should install automatically once downloaded. If it does not, open File Explorer, launch iCloud Setup and restart your PC.
Setting up the app will create iCloud Photos and iCloud Drive folders on your Windows 10 device. These will sync with your iCloud account and connected devices, and should automatically appear in File Explorer’s Quick Access menu.
What are the system requirements?
The current iteration of iCloud for Windows requires that you are running at least Windows 10 and that you re signed into your Microsoft account. You’ll also need Outlook and a web browser (Microsoft Edge, Firefox, Chrome).
If you use Windows 7 or Windows 8, you can download compatible versions of iCloud for Windows directly from the Apple website. (Some iCloud services might work differently, as detailed here.)
Once the software is installed and open, you’ll be asked to enter your Apple ID to sign into iCloud, and you’ll then need to choose which iCloud features you want to use. To do this, you’ll be shown a window in which you check or uncheck each feature:
iCloud Drive,
Photos & videos, including sharing albums of images and videos.
Mail, Contacts, Calendars, Tasks
Bookmarks.
As you enable each of these categories, iCloud for Windows will create folders for those categories in File Explorer, so you can dive in to access or add to your iCloud content. You can also share and collaborate on files held in your iCloud Drive from within File Explorer, with edits synced across your devices.
Here is how to use each of these features:
How does iCloud Photos work?
When enabled, iCloud for Windows creates an iCloud Photos folder in File Explorer. Any images of videos you add to this folder will then be synced across all the devices you own that logged into this Apple ID. You can share galleries of images using iCloud Photo Sharing.
iCloud for Windows will keep lightweight copies of your images on your PC, only downloading full-res versions on request.
When you set yourself up with iCloud for Windows you’ll see an ‘Options’ item appear beside Photos. Tap this and you’ll find the following choices:
iCloud Photo Library: Enable this and any images/videos saved to the iCloud Photos folder on your Windows device will be stored in iCloud. You can also change which folders are used for photo and video sync.
My Photo Stream: Downloads all your most recent images from your devices to your Windows system.
Download new photos and videos to my PC: This option appears when you enable iCloud Photo Library, and is self explanatory.
Keep high-efficiency originals if available: Check this if you have plenty of drive space.
Upload new photos and videos from my PC: Disable this if you only want to view items captured by your other iCloud devices.
iCloud Photo Sharing: You can view items shared with you by others.
Mail, Contacts, Calendars, Tasks and Bookmarks
The final selection of iCloud items you can access using Windows. Once you enable this all your iCloud mail (typically sent to yourname@icloud.com) will be made available in the folder pane on Outlook. You’ll also find all your other calendaring data there, and bookmarks will be made available through your browser.
Apple recently made an extension available for the Chrome browser on Windows to ensure it remains compatible with iCloud and iCloud for Windows. Once the new version is installed, users should find a new “Passwords” section in the app with an iCloud Keychain logo.
Everything else
If you have any problems with these features, please refer to Apple’s extensive Support pages that should help you resolve them. (Don’t forget, you won’t receive email if you failed to create an iCloud email address when setting up your Apple ID).
There are some iCloud services that just don’t work on Windows: Find My iPhone, Back to My Mac, Backup, Notes and Reading List. All these require deeper OS integration, though in the case of Find My iPhone you will be able to track lost devices using iCloud in your browser.
However, if you have valuable data that you want to sync between your Macs, iOS devices and Windows PC, iCloud for Windows should help – though that 5GB of storage Apple supplies remains miserly at best.
(Find out more on how to synchronize iCloud — and what to do when it won’t sync.)
Can I use a Managed Apple ID?
Unfortunately, iCloud for Windows isn’t yet supported if you use a Managed Apple ID.
More useful iCloud tips
Here is a short collection of what may be useful tips for iCloud users on any platform, not just Windows:
Got a story? Please drop me a line via Twitter and let me know. I’d like it if you chose to follow me on Twitter so I can let you know about new articles I publish and reports I find.
Never underestimate the extent to which Apple technology is gaining ground across industries once seen as closed to it. Things have changed — even grocery stores use Apple products today (and not just to take your money).
If you shop at big US food retailers such as Albertsons, Safeway, or Shaw’s, you may already have been close to an Apple product. That’s because these chains rely on an AI-augmented store-ordering solution from a company called Afresh to minimize food waste across meat and seafood departments. Grocery Dive informs us that the tech is being deployed at more than 2,200 stores.
Apple at a grocery store near you
What Afresh does is provide an ordering interface supplemented by AI-powered recommendations and guidance to manage specific food products. The idea is that by combining better product care with AI-supported ordering systems, food is kept fresher, less is unprofitably wasted, and ordering systems become streamlined. The system runs, of course, on iPads, and while it doesn’t replace high standards of food storage and hygiene, if those two things are in place, it can help optimize results.
Thousands of iPads and iPhones are also used at more than 400 supermarkets from top US grocery retailer H-E-B, (hat tip to Brian Wayman for noting this). Equipped with custom apps, these are used to improve how food is sold and stored and to help reduce waste.
The implementation is interesting
When orders are put in, store employees gather the items, scanning them with the iPhone camera as they go; drivers are provided with optimized delivery routes, while iPads are linked to essential sensors, such as temperature sensors, using Bluetooth.
This lets one iPad be used as a mobile control monitoring system to track product storage temperatures across a store, including on shelves or rotisserie; the system also alerts staff when something needs to be put in the fridge to extend its storage. Finally, the Apple system is used with software from Salesforce to help optimize management functions, such as shift schedules, and for maintenance requests, product feedback and more.
Cupertino’s retail wave
UK retailer John Lewis and Partners is another recognized store brand that uses Apple products within its operations. All its staffers have iPhones equipped with the John Lewis app, which helps in terms of inventory management, customer guidance, and customer communication. Back in the US, independent grocers DeCicco & Sons, Cardiff Seaside Market, Village Market, Fruit Center Marketplace and Caraluzzi’s Markets use FoodStorm’s iOS compatible system to manage their own produce.
Elsewhere in the retail sector, Rituals, Hästens, Laura, and many other stores now use Apple devices at point of sale, as inventory management tools, for pick and collect services, deliveries, warehousing, and more.
Apple, the mobile company
There’s a simple reason Apple’s hardware has picked up more presence in retail; it’s the combination of a powerful and highly secure development ecosystem along with mobile devices available in a myriad of formats. The Mac, iPad, iPhone, Apple Watch — even visionOS systems — might be suitable to handle some back-end functions across multiple enterprises.
Speaking during Apple’s Q1 FY24 financial call, Apple CFO Luca Maestri confirmed that the company’s reach into industries it might never have made a dent in before has become an opportunity for the Vision Pro. “We are seeing strong excitement in enterprise. Leading organizations across many industries, such as Walmart, Nike, Vanguard, Striker, Bloomberg and SAP, have started leveraging and investing in Apple Vision Pro as the new platform to bring innovative spatial computing experiences to their customers and employees.”
Now I don’t anticipate we’ll be meeting customer service representatives wearing Vision Pro headsets anytime soon. But as the cost of visionOS devices falls over time and more discreet headsets appear, it’s entirely possible that experiences of that kind will become normalized. And at the current pace of change, this aspect of the digital transformation might not be terribly far away.
The transformations will continue until morale improves
Apple Intelligence could form another string to the company’s bow, as developers of narrow domain-focused AI systems will want to ensure they build in support as seamlessly as possible for enterprise-focused apps on Apple devices. That means thinking about App Intents on the shop floor, or AI as a service in the same way as OpenAI is to be supported on iOS 18.
Such systems could provide contextually relevant information in the moment to optimize store management functions. And, at the current trajectory, they might well carry an Apple logo as they do.
This simply would not have been thought even plausibly possible just a decade ago.
Though it’s a relatively new title, the role of chief artificial intelligence officer (CAIO) is gaining prominence at organizations deploying generative AI (genAI) technology — whether they’re moving deliberately or plowing ahead quickly.
By last October, 11% of midsize and large organizations had already filled a CAIO role, according to research firm IDC — and another 21% were actively seeking one. Just over half of 97 CIOs surveyed last fall said their organization had plans to have an individual leader responsible for AI and about half of those CIOs expect that person to be part of the C-Suite, IDC said.
Newly hired or appointed CAIOs “are not only part of an organization’s C-suite, but they are expected to be one of the most strategic members of the organization,” IDC said in its report.
IDC
As organizations chase efficiency and the productivity promise of AI, the CAIO title is expected to emerge on LinkedIn and other social media feeds, according to Forrester Research Analyst Zeid Khater. In fact, the role could soon surface in one out of eight executive leadership teams.
In a recent Forrester survey, 12% of companies said their CAIO is primarily responsible for the overall enterprise AI strategy; only 2% attributed that responsibility to a chief data officer (CDO). “This doesn’t mean that CDOs are on the verge of extinction,” Khater wrote in a blog post. “Data is still a vital and often unleveraged resource within organizations due to challenges around quality, governance, and access.”
He urged companies to “ensure your AI and data leaders are in lockstep to spin data straw into insights gold. The CAIO brings technical knowledge, while the CDO provides quality data. It’s a powerful partnership for AI success.”
One big factor every CAIO will have to consider is cost; deploying AI models is expensive because cloud providers and proprietary genAI use cases require a lot of computing power — high-end, expensive computing power. And the chips that power learning and inference processes in large language models can cost thousands of dollars. (Nvidia makes most of the GPUs for the AI industry, and its primary data center workhorse chip costs $10,000; the company’s lock on the AI chip market is, however, being challenged by others who hope to undercut it with lower chip prices.)
All federal agencies will have CAIOs
It’s not just private companies looking to hire. In March, US President Joseph R. Biden Jr. gave all federal agencies two months to appoint CAIOs who be responsible for promoting AI innovation, coordinating with other agencies, and managing risks associated with the technology. The 60-day deadline highlighted the urgent need for governance as AI continues its meteoric adoption.
“While AI is improving operations and service delivery across the Federal Government, agencies must effectively manage its use,” Biden’s memo said. “The risks… esult from any reliance on AI outputs to inform, influence, decide, or execute agency decisions or actions, which could undermine the efficacy, safety, equitableness, fairness, transparency, accountability, appropriateness, or lawfulness of such decisions or actions.”
Twenty-four federal agencies had appointed CAIOs by the May 30 deadline. In all, the Biden administration plans to hire 100 AI professionals by this summer and is requiring all federal agencies to establish AI governance boards to coordinate adoption efforts and establish rules for the use of AI and genAI.
“The decision to institutionalize the role of CAIOs demonstrates a clear acknowledgment of AI’s strategic significance,” Joel Meyer, former deputy assistant secretary of the US Department of Homeland Security, said in a recent Fedscoop article. (Meyer led the creation of DHS’s AI Task Force.)
Lt. Gen. John Shanahan, who co-authored the article with Meyer and was the nation’s first director of the Department of Defense Joint Artificial Intelligence Center, said one CAIO responsibility “is to identify low-hanging fruit. AI pilots can be chosen thoughtfully to demonstrate hypotheses that can then be affirmed in each department’s AI strategy. These quick wins can build momentum for broader AI strategy implementation.”
Because federal agencies were given latitude to define the organization under CAIOs, there’s a lot of variety between them in terms of authority, budgets, and what how the role would be executed, according to Amy Jones, US Public Sector AI Market Lead with Ernst & Young.
“Day to day responsibilities [are] pretty varied,” she said. “I think a CAIO’s success would be agency literacy. We all use the internet and email every day, and that requires literacy on both how to use them safely and securely and also how to use them optimally.”
The same is true for genAI technology.
IDC
Data quality matters
One known hurdle for genAI rollouts is the quality of data used to train LLMs. As the saying goes: garbage in, garbage out. It’s both challenging and costly to obtain high-quality, unbiased, and representative data, according to Andrew Rabinovich, who recently took the new position as Head of AI at freelance job platform Upwork.
According Robinovich, key considerations for new CAIOs looking to deploy AI include:
Careful planning and consideration for how the technology will deliver real customer impact rather than moving forward just for the sake of it or to keep pace with the hype cycle.
A clear understanding of business objectives and specific customer pain points to solve with AI before launch.
Evaluating and ensuring the quality and reliability of the AI models being created, whether home-grown or by a third-party provider.
Ensuring that LLMs are trained on diverse and representative datasets to avoid bias, while consistently monitoring for iterative improvements.
“Ensuring data cleanliness and accuracy often requires extensive pre-processing, which is both time-consuming and resource-intensive — and that’s if you even have access to the right datasets,” Robinovich said.
CAIOs and others tasked with overseeing AI deployments play an essential role in “shaping an organization’s strategic, informed and responsible use of AI,” he said. “There are many responsibilities baked into the role, but at its core, it’s about steering the direction of AI initiatives and innovation to align with company goals. AI leads must also create a culture of collaboration and continuous learning.”
“All teams across all functions within an organization should be thinking about how they can collaborate on AI projects, experiment with the technology and explore how to equip their teams with the right knowledge, skills and tools to harness AI,” he said.
IDC
At Upwork, Robinovich is overseeing the company’s use of the GPT-4 LLM to create an AI-powered platform called Uma It will power a “Best Match” job search site (currently in beta); a job post generator to create job post drafts for employers seeing candidates; the creation of job proposal tips to surface the most applicable skills and write proposals for them; and an Upwork Chat Pro designed to help internal employees complete repetitive tasks faster.
“We’re developing Uma to underpin our entire platform and, increasingly, serve as a conversational Upwork companion to our customers,” Rabinovich said.
For genAI to function reliably, CAIOs will need to figure out how to utilize AI and data optimization techniques for improved efficiency, data quality, and ethical considerations. “On paper, [you need] baseline compliance — making sure they [LLMs and genAI] are within regulatory and policies, creating [your] own policy within the agency that’s specific to the mission, [and] identification of inventory of use cases,” Jones said.
Robinovich agreed. The quality of data used to train AI models is an important aspect of the development process, but it can be hard to obtain high-quality, unbiased, and representative data, he noted.
“Ensuring data cleanliness and accuracy often requires extensive pre-processing, which is both time-consuming and resource-intensive — and that’s if you even have access to the right datasets,” Robinovich said.
A dedicated CAIO or one with shared duties?
Jenn Kosar, a partner at PricewaterhouseCoopers (PwC), said while most organizations have not yet designated CAIOs as an official C-suite role and title, from a functional perspective a significant number of organizations are filling the role today without the title. Most often, the position is one notch below a CIO, she said.
“Today, we often seen CTOs and CISOs taking this [genAI responsibility] on,” Kosar said. “And that may be OK for where we are today. But the strategic [planning], the change management, the innovation, the ability to take an organization through a transformation — these are really critical skills to the success of this role.
“Unfortunately, what we’re seeing in most instances it’s not a full-time job. In other words, they [CAIOs] have other roles. We believe it should be a dedicated role. They’re being held accountable for how an organization is moving forward with AI.”
While CAIOs might not always be seated at the C-suite table, those who are there are keenly focused on genAI and its potential to drive efficiencies and profits. Without an executive guiding those deployments, achieving the performance and ROI organizations seek will be tough, she said.
“It’s hard to imagine how pieces come together and how you’d bring together so many players,” Kosar said, noting that PwC has more than a dozen different LLMs running internally to power AI tools and products in virtually every business unit.
“You have to have the ability to do short-term and long-term planning and balance the two and stay focused on innovation,” she continued. “At the same time, you need to recognize the pace of change while not getting distracted by the latest shiny object.”
Getting AI right is important because of how much it will be a part of everyday life by the end of the decade, Robinovich said. By 2030, he believes virtually everyone will interact with AI and the tech will perform in roles varying from personal assistants and tutors to therapists and accountants — even lawyers.
“AI will help humans uplevel and enhance societies, because it’ll enable humans to focus on solving ever more complex problems,” Robinovich said.
Microsoft this week released 49 updates (including two recent additions) on Patch Tuesday with no reported zero-day flaws, public disclosures, or newly released working exploits for the Microsoft ecosystem. This came as welcome news and is paired with low-risk changes to Microsoft Office. The company’s development platforms saw minor updates to Visual Studio, and both SQL Server and Microsoft Exchange were patch free for the month.
The team at Readiness has provided a useful infographic outlining the risks associated with each of the updates.
Known issues
Each month, Microsoft publishes a list of known issues that are part of the latest update cycle, including the following reported minor issues:
After you installKB5034203 (dated 01/23/2024) or later updates, some Windows devices that use the DHCP Option 235 to discover Microsoft Connected Cache (MCC) nodes in their network might be unable to use those nodes. Microsoft is still working on this one. In the meantime there is a workaround that involves setting the Cache Hostname to 1.
We recognize and respect Microsoft’s recent efforts with artificial intelligence (note, I did not say “AI” as that is an Apple thing now) but it would be nice if Microsoft resolved theprofile picture (that you can’t change) known issue soon.
Major revisions
Microsoft published the following major revisions to past security and feature updates including:
CVE-2024-30080: (see below for mitigations). This patch was updated late in the June release cycle. As this was an information update, no further action is required, unless you want to action the Microsoft recommended mitigations.
Mitigations and workarounds
Microsoft published the following vulnerability-related mitigations:
CVE-2024-30070: DHCP Server Service Denial of Service Vulnerability. Microsoft (helpfully) notes that if you’re not using DHCP, you are not affected by this potential vector for DDOS attacks.
CVE-2024-30080: Microsoft Message Queuing (MSMQ) Remote Code Execution Vulnerability. Message Queuing security issues are tough to find, mitigate and test, so this might need some careful attention from your internal developers. At the very least, ensure that you have changed your ports from the MSMQ listening default (1801) to help reduce your attack surface. Microsoft also recommends you check to see whether the MSMQ HTTP-Support feature is enabled.
The team at Readiness analyzed the latest Patch Tuesday updates to provide detailed, actionable testing guidance based on assessing a large application portfolio and a detailed analysis of the Microsoft patches and their potential impact.
For this cycle, we have grouped the critical updates and required testing efforts into different functional areas including:
Microsoft Office
Microsoft SharePoint will require basic document opening and multi-user access tests this month.
Microsoft .NET and Developer Tools
There are no updates to Microsoft .NET requiring application portfolio testing this month.
Windows
The following core Microsoft features have been updated:
Changes to Secure Boot will require testing of all third-party drivers.
Code integrity policies need to be verified for Windows Lockdown (WLDP), Windows Defender Application Guard (WDAG) and theWindows Driver Policy for Intune deployments. We recommend you test your Windows desktop sandbox and ensure that it boots correctly.
Changes to Windows networking will require testing at least twoDHCP servers.
Remote desktop-related updates will require VPN connection tests. Try some administrative commands from the Microsoft Management console (MMC) such as adding, connecting and disconnecting VPN connections.
This month’s update also affects several core systems such as Kernel32 and Win32K.SYS sub-systems. Unfortunately, these changes affect how applications behave at a fundamental level, which makes testing not just hard, but broad and expansive across your application portfolio. The Readiness team suggests that the following general application tests be performed against all of your core line-of-business applications.
Test as many windows and pop-ups as possible.
Check window title bars for errors, or poorly formatted text.
Check for unusual items in the Windows taskbar.
Thoroughly test File explorer (sorry about that).
Test multiple applications, with multiple windows.
Automated testing will help with these scenarios (especially a testing platform that offers a “delta” or comparison between builds). However, for your line-of-business apps, getting the application owner (doing UAT) to test and approve the results is essential.
Windows lifecycle update
This section contains important changes to servicing (and most security updates) to Windows desktop and server platforms.
Windows 10 Enterprise and Education, Version 21H2 will no longer be serviced as of June 11, 2024
For those planning ahead, Oct. 8, 2024, is a big day as Microsoft will no longer offer general servicing for the following desktop platforms:
Windows 11 Enterprise and Education, Version 21H2
Windows 11 Home and Pro, Version 22H2
Windows 11 IoT Enterprise, Version 21H2
Each month, we break down the update cycle into product families (as defined by Microsoft) with the following basic groupings:
Browsers (Microsoft IE and Edge)
Microsoft Windows (both desktop and server)
Microsoft Office
Microsoft Exchange Server
Microsoft Development platforms (ASP.NET Core, .NET Core and Chakra Core)
Adobe (if you get this far)
Browsers
Microsoft has released seven minor updates to the Chromium-based browser (Edge), while theChromium project has added six additional updates this week. These updates should have minor to negligible impact on applications that integrate and operate on Chromium. Add these updates to your standard patch release schedule.
Windows
This month, Microsoft released one critical update (CVE-2024-30080) and 32 patches rated as important for Windows, covering the following key components:
Windows Win32 Kernel Subsystem, GRFX and drivers
Networking (Wii-fi) and DHCP
Storage and Error Reporting
Crypto and BitLocker
The critical-rated patch relates to the core, but not often used, Message Queuing service (MSMQ) that could affect internal applications. Unusually, this patch has already been updated since the main release on Tuesday. That said, the Readiness team believes all these Windows patches can be added to your standard release schedule.
Microsoft Office
There were no critical updates for Office this month, and only five patches rated as important. All five have low potential for exploitability (no worms, add-in vulnerabilities or Word macro issues) and should be added to your regular Microsoft Office update schedule.
Microsoft Exchange Server
No updates for Microsoft Exchange Server or SQL Server this month, which, of course, is a good thing.
Microsoft development platforms
Microsoft released just three updates to Microsoft Visual Studio. These patches affect versions of the Microsoft developer platform from 2017 to 2022. All of the proposed changes are low risk and application specific. Add these updates to your standard developer release schedule.
Adobe Reader (if you get this far)
We are back to the usual state of things, and Microsoft has not chosen to include any Adobe products this release cycle. This is a very good thing.
Microsoft this week released 49 updates (including two recent additions) on Patch Tuesday with no reported zero-day flaws, public disclosures, or newly released working exploits for the Microsoft ecosystem. This came as welcome news and is paired with low-risk changes to Microsoft Office. The company’s development platforms saw minor updates to Visual Studio, and both SQL Server and Microsoft Exchange were patch free for the month.
The team at Readiness has provided a useful infographic outlining the risks associated with each of the updates.
Known issues
Each month, Microsoft publishes a list of known issues that are part of the latest update cycle, including the following reported minor issues:
After you installKB5034203 (dated 01/23/2024) or later updates, some Windows devices that use the DHCP Option 235 to discover Microsoft Connected Cache (MCC) nodes in their network might be unable to use those nodes. Microsoft is still working on this one. In the meantime there is a workaround that involves setting the Cache Hostname to 1.
We recognize and respect Microsoft’s recent efforts with artificial intelligence (note, I did not say “AI” as that is an Apple thing now) but it would be nice if Microsoft resolved theprofile picture (that you can’t change) known issue soon.
Major revisions
Microsoft published the following major revisions to past security and feature updates including:
CVE-2024-30080: (see below for mitigations). This patch was updated late in the June release cycle. As this was an information update, no further action is required, unless you want to action the Microsoft recommended mitigations.
Mitigations and workarounds
Microsoft published the following vulnerability-related mitigations:
CVE-2024-30070: DHCP Server Service Denial of Service Vulnerability. Microsoft (helpfully) notes that if you’re not using DHCP, you are not affected by this potential vector for DDOS attacks.
CVE-2024-30080: Microsoft Message Queuing (MSMQ) Remote Code Execution Vulnerability. Message Queuing security issues are tough to find, mitigate and test, so this might need some careful attention from your internal developers. At the very least, ensure that you have changed your ports from the MSMQ listening default (1801) to help reduce your attack surface. Microsoft also recommends you check to see whether the MSMQ HTTP-Support feature is enabled.
The team at Readiness analyzed the latest Patch Tuesday updates to provide detailed, actionable testing guidance based on assessing a large application portfolio and a detailed analysis of the Microsoft patches and their potential impact.
For this cycle, we have grouped the critical updates and required testing efforts into different functional areas including:
Microsoft Office
Microsoft SharePoint will require basic document opening and multi-user access tests this month.
Microsoft .NET and Developer Tools
There are no updates to Microsoft .NET requiring application portfolio testing this month.
Windows
The following core Microsoft features have been updated:
Changes to Secure Boot will require testing of all third-party drivers.
Code integrity policies need to be verified for Windows Lockdown (WLDP), Windows Defender Application Guard (WDAG) and theWindows Driver Policy for Intune deployments. We recommend you test your Windows desktop sandbox and ensure that it boots correctly.
Changes to Windows networking will require testing at least twoDHCP servers.
Remote desktop-related updates will require VPN connection tests. Try some administrative commands from the Microsoft Management console (MMC) such as adding, connecting and disconnecting VPN connections.
This month’s update also affects several core systems such as Kernel32 and Win32K.SYS sub-systems. Unfortunately, these changes affect how applications behave at a fundamental level, which makes testing not just hard, but broad and expansive across your application portfolio. The Readiness team suggests that the following general application tests be performed against all of your core line-of-business applications.
Test as many windows and pop-ups as possible.
Check window title bars for errors, or poorly formatted text.
Check for unusual items in the Windows taskbar.
Thoroughly test File explorer (sorry about that).
Test multiple applications, with multiple windows.
Automated testing will help with these scenarios (especially a testing platform that offers a “delta” or comparison between builds). However, for your line-of-business apps, getting the application owner (doing UAT) to test and approve the results is essential.
Windows lifecycle update
This section contains important changes to servicing (and most security updates) to Windows desktop and server platforms.
Windows 10 Enterprise and Education, Version 21H2 will no longer be serviced as of June 11, 2024
For those planning ahead, Oct. 8, 2024, is a big day as Microsoft will no longer offer general servicing for the following desktop platforms:
Windows 11 Enterprise and Education, Version 21H2
Windows 11 Home and Pro, Version 22H2
Windows 11 IoT Enterprise, Version 21H2
Each month, we break down the update cycle into product families (as defined by Microsoft) with the following basic groupings:
Browsers (Microsoft IE and Edge)
Microsoft Windows (both desktop and server)
Microsoft Office
Microsoft Exchange Server
Microsoft Development platforms (ASP.NET Core, .NET Core and Chakra Core)
Adobe (if you get this far)
Browsers
Microsoft has released seven minor updates to the Chromium-based browser (Edge), while theChromium project has added six additional updates this week. These updates should have minor to negligible impact on applications that integrate and operate on Chromium. Add these updates to your standard patch release schedule.
Windows
This month, Microsoft released one critical update (CVE-2024-30080) and 32 patches rated as important for Windows, covering the following key components:
Windows Win32 Kernel Subsystem, GRFX and drivers
Networking (Wii-fi) and DHCP
Storage and Error Reporting
Crypto and BitLocker
The critical-rated patch relates to the core, but not often used, Message Queuing service (MSMQ) that could affect internal applications. Unusually, this patch has already been updated since the main release on Tuesday. That said, the Readiness team believes all these Windows patches can be added to your standard release schedule.
Microsoft Office
There were no critical updates for Office this month, and only five patches rated as important. All five have low potential for exploitability (no worms, add-in vulnerabilities or Word macro issues) and should be added to your regular Microsoft Office update schedule.
Microsoft Exchange Server
No updates for Microsoft Exchange Server or SQL Server this month, which, of course, is a good thing.
Microsoft development platforms
Microsoft released just three updates to Microsoft Visual Studio. These patches affect versions of the Microsoft developer platform from 2017 to 2022. All of the proposed changes are low risk and application specific. Add these updates to your standard developer release schedule.
Adobe Reader (if you get this far)
We are back to the usual state of things, and Microsoft has not chosen to include any Adobe products this release cycle. This is a very good thing.
Apple Vice President Craig Federighi calls Apple Intelligence, “the personal intelligence system that puts powerful generative models right at the core of your iPhone, iPad, and Mac.”
To achieve this, it draws on what your device knows about you and on-device intelligence, or, where necessary, in the cloud via the highly secure Private Cloud Compute system. At all times, Apple says it’s working to protect user privacy, which means your data is protected unless you choose to use a third-party AI, such as ChatGPT.
In making these solutions, Apple has paid particular heed to creating tools that offer truly useful help. The critical idea is that they get things done for you without getting in the way of the easy user interaction you usually enjoy with your Apple product.
Tools to help you write better
To help you write, Apple Intelligence can proofread and rewrite your text anywhere across your system, including within third-party apps. Apple Intelligence will also summarize a meeting transcript, long email, and website content; pretty much any large block of text can be made bite-sized.
Be warned, for some of these functions Apple Intelligence might need to use ChatGPT, but you’ll be told if that is the case and can cancel the request rather than sharing your information with a third-party service provider.
The system works to figure out which of your incoming emails are most essential and places those emails at the top of your Inbox.
AI will also create what it thinks are appropriate replies for you — you don’t have to use them and do get to approve them before they are despatched.
Meetings, now with AI assistants
Tap record when making a call or when inside a Note to capture audio recordings and transcripts. Once the call or meeting ends, Apple Intelligence will quickly generate a summary of the transcript.
Tools to help you stay focused
There’s a new automated Focus mode that reduces interruptions but is also intelligent enough to let important notifications break through. Apple Intelligence will also get to know which of your notifications matter to you most and make sure those are at the top of your notifications list. The idea is to optimize your attention so you can stay on top of the things.
Making images
Apple’s on-device LLM engine will create original images for you based on a typed request. Usefully, it will also remove unwanted objects in an image on request. And a new Image Playground app lets you experiment with ideas and try different image styles to create your own images.
Photos gets better at helping you find your stuff
AI features in Photos include far more powerful and contextually-aware Search results and the ability to create a Memories video based on such a search.
Introducing, Genmoji
If like me you have problems finding precisely the right emoji or aren’t really certain if any that you do choose to have a double meaning, then salvation is at hand! Genmoji makes it possible to create completely original emoji on demand; just tell your Apple device what you want and up it will pop.
Wave your Image Wand
This feature needs an Apple Pencil. It works like this: Open a Note, draw a circle where you want your generated image to appear in that Note, and Apple’s intelligence will make you a custom image that reflects the contents of it.
Siri gets serious attention
We’re being promised lots of improvements in Siri; not only will it be able to better understand more complex or poorly articulated requests, but it also gains the kind of contextual understanding you need to figure out answers to complex questions such as “Show me the recipe Sacha sent me the other day.”
That has several implications, including:
Siri knows what you are looking at and you can make requests that reference that, such as adding an address to your Contacts, or adding something to a note in a different app.
The assistant can also now answer questions about any of your Apple devices or operating system features, like an Apple Genius in your pocket.
Siri also now understands typed requests — double tap the bottom of the display and a keyboard pops up.
Sometimes your device might need to use ChatGPT to fulfill some requests; you will be told if that is the case and can cancel the request.
Apple has also given asking Siri questions a new vibe; when you do so, your device now will show a glowing light all around the borders of the screen.
Is there more to come? Probably
It is likely there will be additional features in place by the time Apple Intelligence is made available in the fall product software updates. This is because developers can use App Intents to make features available within their apps also available across the system. Meanwhile, developers get to use Xcode Complete to work smarter.
Apple is also thought to be pushing other genAI firms beyond OpenAI to offer their services on its platforms, while the company hopes to generate new income streams as developers build and make available additional LLMs on its platforms.
