Month: August 2024

Where are my AR glasses?

August 9, 2024 / Security – Computerworld

META founder and CEO Mark Zuckerberg recently said that hundreds of millions of people might wear AR glasses. (He was speaking with Nvidia CEO Jensen Huang at this year’s SIGGRAPH conference.)

I have to say, I agree; I’ve made similar predictions in this space during the past couple of years. I think AR glasses —first without, then with, holographic images projected onto the lenses — will be the next big thing in consumer technology.

I also agree with Zuckerberg’s claimed approach to the category. During that same conversation, he said: “Let’s constrain the form factor to just something that looks great. And within that, let’s put in as much technology as we can.”

That’s the opposite approach of most AR glasses makers. TCL RayNeo X2, Vuzix Ultralite, Rokid Max, XREAL Air and others start with: What’s the best visual experience we can ship within a reasonable price? They sacrifice appearance for quality imagery and lower price, but it’s a fatal sacrifice for mainstream acceptance.

The result tends to be something that’s great to use but which nobody wants to be seen wearing outside.

As Google learned with Google Glass, socially unacceptable glasses will never go mainstream.

Ray-Ban Meta glasses, meanwhile, Meta’s only market success in hardware ever, follows the Zuckerberg model. (Zuckerberg claimed on a recent earnings called that Ray-Ban Meta “demand is still outpacing our ability to build them.”) The glasses look like normal glasses. And to make that work within a low price (starting at $300) there is no visual element in the lens. All output is audio. The camera can process multimodal input (photos, not video), but there is no light engine, no special lenses and no need for a bigger battery.

Still, Meta is clearly working on holographic visual AR glasses. The company is working on custom chips and partnering with Luxottica on getting the form factor right. Rumors circulating in Silicon Valley say Meta could publicly demonstrate AR glasses as early as October.

Another interesting player is Brilliant Labs, which sells its Frame AI glasses. In theory, these sound fantastic. The glasses feature a microOLED display with a 20-degree diagonal field of view in the right eye. Frame accesses familiar generative AI chatbots like ChatGPT, Whisper and Perplexity. A forward-facing camera enables live translation, visual analysis and internet queries. The open-source design allows developers to customize and enhance the glasses’ functionality using provided tools and plugins. And they’re surprisingly inexpensive: $349 for standard lenses, $448 for prescription lenses.

Frames have clear downsides, as well. They lack built-in speakers and require connection to a smartphone for full functionality. Battery life ranges from two to six hours. But the biggest downside is their appearance. While they’re in the ballpark of ordinary looking glasses, the round frames stand out and draw attention in a bad way. While interesting for curious makers and tinkerers, the combination of poor battery life and dorky appearance make it clear that Frames glasses are not something an office professional could wear at work.

Both startups and major tech companies are in a hot race to get to market with AR/AI glasses that look like regular glasses. That includes Apple, Google, Microsoft and dozens of other companies.

Which raises the questions: Where are the glasses? Why is it taking so long?

Components are too big, power-hungry and expensive

It’s possible right now to build great AR glasses. They would look like regular glasses, project holographic images anchored in physical space. And a camera would hoover up video for multi-modal input to advanced AI. That’s the good news.

The bad news is that the battery would last maybe 45 minutes and they would cost oh, say, $10,000 a pair.

I’m making those numbers up. The point is that we have the technology to create great AI glasses, but need component shrinking, cost reductions and power efficiency on a whole new scale to make them viable in the market.

Huge strides have been made in the miniaturization of components, but more work remains. AR glasses need to fit all those electronic components into a regular-size frame. Even more difficult is keeping the weight down.

And while glasses must be made smaller and lighter, batteries must be bigger and more powerful.

Even more challenging: Batteries need high energy density to provide sufficient power for the displays, processors and sensors in a compact form factor. Heat management is also an engineering challenge — the batteries can’t get hot because they’ll be right up against users’ temples. Companies are exploring advanced materials, like solid-state electrolytes and nano-materials. Big benefits could come from flexible and curved batteries for better integration into eyeglass frames. And technologies like solar cells or kinetic energy harvesting could help extend battery life.

There are also qualitative hurdles to overcome. Light engines, which are the part of AR glasses that projects images onto lenses, tend to suffer from light leakage (where other people can see your screen and your glasses “light up” in low light), ghosting, rainbow artifacts, low resolution and more.

What’s interesting about the light engine component industry is that the major players — a group that includes Avegant, VitreaLab, Lumus and TriLite Technologies — are all working on the same problems, but with radically different approaches. For example, Avegant’s use of various display technologies and VCSEL contrasts with VitreaLab’s focus on quantum photonics and 3D waveguides. Lumus’s reflective waveguide technology differs from both, offering a unique method of image projection. TriLite’s laser beam scanning technology represents yet another distinct approach.

It will be interesting to see how these approaches shake out, and which approach offers the best combination of price, performance and size and weight.

So when do we all get all-day, everywhere AR glasses?

Following Zuckerberg’s maxim — “Let’s constrain the form factor to just something that looks great. And within that, let’s put in as much technology as we can” — we could see something creative from a major player soon.

As we learned with Ray-Ban Meta glasses, by making the right trade-offs, it’s possible to get a great, wearable product at low cost. The key now is adding a holographic display.

One cost-cutting measure will be a display in one eye instead of two. Also: By offering visual elements sparingly, and mainly focusing on an audio interface, battery problems might be solved.

Another possibility — what if the display information showed only text and not pictures? I think most people would enjoy what might look like subtitles, offering language translation, contextual information, turn-by-turn directions and other information. Pictures and graphics can wait, if that improves battery life and cuts down on light engine problems like light leakage.

Another shortcut is to offer just a heads-up display, rather than a display showing text and objects anchored in physical space — like Google Glass rather than Apple Vision Pro.

And yet another point to consider is that AR glasses with holographic image capability don’t have to be as inexpensive as todays audio-only AI glasses. Ray-Ban Metas start at $300, but the right price for a great pair of AR glasses might be as much as $1,000.

The bottom line is that amazing AR glasses that look like ordinary eyeglasses are still coming. But truly high-quality, no-compromise devices won’t arrive anytime soon. It make take five years for more advanced developments in batteries, light engines, lenses and other components to be available at reasonable prices.

In the meantime, the platform will benefit from creative trade-offs that provide something useful and appealing, though not perfect.

With the right combination components, persistent access to AI and glasses people really want to wear in public, Zuckerberg’s predictions about hundreds of millions of people wearing AR glasses might well turn out to be actually conservative.

8 out-of-sight superpowers for Google Contacts on Android

August 9, 2024 / Security – Computerworld

Quick: What’s the most exciting app on your Android phone right now?

Just a hunch here, but I’m gonna go out on a limb and say Google Contacts probably wasn’t your answer. And why would it be? Your phone’s virtual Rolodex is about as exhilarating as a trip to the endodontist. Plus, our mobile devices have had systems for managing our contacts since way back in the prehistoric days, so it certainly doesn’t seem like something to celebrate.

Hold the phone, though — ’cause Android’s current contacts setup is much more than just a dusty ol’ place to dump names and numbers. The Google Contacts app has some genuinely useful advanced options that can make your life easier and make your phone more intelligent. And all you’ve gotta do is dig ’em up and start putting ’em to use.

One important note, before we dive in: All of these tips revolve around the Google Contacts app, which is the default Android contacts app for Google’s own Pixel phones and certain other devices. If you’re using a Samsung Android phone or any other device where the manufacturer swapped out Google Contacts for its own inferior alternative, I’d strongly suggest taking a moment to switch yourself over.

Aside from allowing you to tap into all of the tricks we’re about to explore, that’ll empower you to keep your contacts continuously synced with your existing Google account and thus be able to access ’em easily from a computer or any other Android device you sign into in the future, no matter who made it — — without any antiquated manual-transfer silliness or other effort required.

Cool? Cool. Let’s get into it.

[Hey: Want even more advanced Android knowledge? Check out my free Android Shortcut Supercourse to learn tons of time-saving tricks for your phone.]

Google Contacts Android trick #1: Group intelligence

One of the Google Contacts app’s most underappreciated possibilities on Android is the way the service lets you organize the humans and/or marsupials you know into different groups and then simplify how you interact with ’em.

Contacts’ grouping system follows the familiar Gmail labels-style approach, in which you can assign any number of custom labels onto different people’s profiles and then group ’em accordingly — while also continuing to see ’em in your main contacts list.

So, for instance, you might use a “Friends and Family” label to cover everyone in your phone who isn’t work-related. You might put a label called “Exceptionally Smart People” onto all of your Android-carrying colleagues and comrades. Or maybe you’d slap a “Squash Buds” label onto all the profiles of your squash-playing pals and/or fellow gourd enthusiasts.

Whatever the case may be, creating groups in Google Contacts gives you two key advantages on Android — sheer satisfaction of organization aside:

It makes it incredibly easy to fire off emails to everyone in the group at once from your phone.

It makes it delightfully simple to start up a new messaging thread with everyone in the group in the Google Android Messages app (or whatever Android texting app you prefer).

To get started, just open up the Google Contacts app on your phone, tap the little label icon in its upper-right corner — the second icon from the right — then tap “New label” to start your first contacts group and decide who should belong to it. You can do the same thing from the Google Contacts website, too, if it seems easier to manage on a computer, and any changes you make in either place will always instantly sync to the other.

Once you’ve got your groups going, the real power comes into play. In the Google Contacts app on Android:

Tap the label icon in the upper-right corner once more — and this time, select any of the labels you’ve created.
Tap the three-dot menu icon in the upper-right corner on the label overview screen that comes up next.
And there, you’ll see options to “Send email” or “Send message” — hands-down the fastest ways to cook up a new email or message thread to any preselected group of alleged humans on your phone.

Google Contacts Android: Labels — The Android Google Contacts app’s labels system holds some supremely useful group management powers.

JR Raphael, IDG

Weirdly enough, you can’t accomplish that same feat directly from Gmail or Google Messages. The Contacts app really is the secret ingredient — and now that you know, you’ll never waste another second thinking about group conversation creation again.

Google Contacts Android trick #2: Reminder minding

I don’t know about you, but my ability to remember important dates — birthdays, anniversaries, and other such milestones — is about as reliable as Motorola’s Android upgrade commitment.

But whether you’re trying to remember a partner’s special day or a client’s anniversary of signing up with your company, the Android Contacts app can go a long way in making sure your mushy mammal brain has the right info at exactly the right time.

Just open up Google Contacts on your phone, tap the Organize tab at the bottom, then tap “Reminders.”

And hey, how ’bout that? Right then and there, you can add a new significant date for any specific person in your contacts and then tell the app exactly how and when you want to be reminded.

Google Contacts Android: Reminders — Reminders are a hidden gem within the Android Google Contacts app.

JR Raphael, IDG

Contacts will alert you as requested, year after year — no additional thought or effort required. Plus, you’ll always see that info as part of the person’s profile within the app, anywhere and anytime you open it.

Google Contacts Android trick #3: Custom defaults

Have you ever gone to call someone and gotten that little pop-up prompt asking you which of their numbers you want to use? It’s a pretty common occurrence when you have people with multiple digits on your device — a cell and work number, a home and monkey cage number, or whatever the case might be.

Or maybe you’ve tried typing someone’s name into Gmail and had the wrong email address show up as a suggestion — a personal address instead of a work account, an old Hotmail address instead of the one they actually check, or any other such oddity.

On the calling front, Android usually offers to let you set a default dialing number for any contact the very first time you call ’em. But if you ever want to change that default after the fact, it’s tough to know where to turn.

And when it comes to email, you’re relying purely on Google’s best guess as to which address you’re likely to want for any given contact. And there’s no obvious way to adjust that judgment.

In both cases, Google Contacts is your answer:

In the Contacts app on Android, find and open the specific contact you want to adjust.
Tap the three-dot icon in the upper-right corner of the screen and select “Set defaults.”
You’ll then be able to pick the default phone number and email address for that person (provided you have more than one of each saved within their profile).

Google Contacts Android: Defaults — Taking the time to set defaults for important calendars can save you precious moments each and every day.

JR Raphael, IDG

Good to know, no?

Google Contacts Android trick #4: Custom ringtones

An oft-overlooked option in recent Android versions is the ability to set a specific custom ringtone for every contact who calls your phone with any regularity. That way, you’ll know within a second who’s ringing you and how urgently you need to answer or ignore it, based solely on the sound coming out of your phone’s speaker.

To create your own custom contact-specific ringtones, just head back into the Google Contacts app on your Android device, then:

Find and open the entry for the person you want to adjust.
Tap the three-dot menu icon in the upper-right corner of the screen and select “Set ringtone.”
Find and select whatever lovely little ditty best suits the person’s personality (e.g. Kenny Loggins’ “Danger Zone” or Pink Floyd’s “Run Like Hell,” for your boss).

And that’s it: The next time that contact calls, you’ll hear whatever sound you chose instead of your standard system ringtone — and you’ll instantly know what level of fear and/or dread to feel.

Google Contacts Android trick #5: A subtle silencer

Got someone on your list who dials your digits a little too often? In addition to setting a custom ringtone, you can go a step further and tell your phone to automatically route calls from any specific number directly into your voicemail — so you never even hear a single ring.

It’s a less aggressive and obvious way to avoid someone’s calls than full-on blocking, which is available within Google’s Android Phone app on any device where it’s installed. And unlike blocking, this approach allows you to receive a message and decide if and when it’s worth returning that hackle-raising human’s call.

Here’s how to do it:

Open up Google Contacts on your phone and once more find and select the person you have in mind.
Scroll all the way down to the bottom of the screen and look for the “Route to voicemail” option.
Tap that option to send all future calls from that contact’s number directly to your voicemail without ever ringing.

And make yourself a mental note, too: If you ever have a change of heart and want to allow that contact’s calls through again, you can find the option to do so in that very same place within the Google Contacts app.

Google Contacts Android trick #6: Instant organization

While we’re thinking about the creatures in your contacts, let’s take 10 seconds to spruce up your setup and streamline any duplicate entries.

You’d be surprised how often that happens — with multiple contacts being saved for the same person over time, as email addresses change and your once-pristine lineup of Somewhat Important Persons (SIPs) grows ever-more messy.

Luckily, Google Contacts is incredibly good about cleaning that up for you, with next to no active effort. Just open up Contacts on your phone, hit the Organize tab at the bottom, and select “Merge & fix.”

Follow the suggestions you’re presented with — including, in particular, any option to “Merge duplicates” (which will show up only if it’s relevant for you at any given moment) — and then bask in your freshly optimized and simplified contacts setup.

Google Contacts Android: Merge & fix — The Google Contacts app’s “Merge & fix” system can work wonders on your contacts organization.

JR Raphael, IDG

Google Contacts Android trick #7: Your location station

Location sharing can be a fantastically handy way to keep tabs on friends, family members, or even sales reps out in the field during the workweek — with everyone’s active knowledge and willing participation, of course.

And while Google Maps is what actually powers all of Android’s location-sharing prowess, the Google Contacts app has gained a helpful bit of integration with that system in recent years.

Specifically, you can now see and manage live, real-time location sharing with anyone right from within the Contacts app on Android. Just open up any individual contact within the app, then tap the Location Sharing icon beneath their name and image to get started.

Once the person agrees to share their location with you, you’ll see a map showing their current physical position right within their contact profile — and, in an especially useful twist, you’ll also see their up-to-the-minute location within the Google Contacts “Individual contact” widget, if you opt to add such a widget for that person onto your home screen.

Speaking of which…

Google Contacts Android trick #8: Easy access

Last but not least in our collection of Contacts tricks is a splendid shortcut for saving seconds when calling or messaging your most common contacts — without having to have a big honkin’ widget on your home screen for every single person you interact with.

It’s part of Android’s woefully underused App Shortcuts system, and it can seriously step up your day-to-day efficiency while also making the most of your home screen real estate.

To set this one up, you’ll want to start on your actual home screen:

First, press and hold any open space.
Find and select the option to add a widget.
Find Google Contacts (likely listed simply as “Contacts”) in the list that comes up. Tap it, and tap it with gusto!
Select either “Direct Dial” or “Direct Message,” then find and select the specific contact you want. (Depending on your device, you might either tap the option or press and hold it to select it.)

That’ll put a one-tap shortcut for calling or texting that person right on your home screen for especially easy access:

Google Contacts Android: Shortcuts — The Google Contacts home screen shortcuts give you one-tap access to important people, anytime — right on your Android home screen.

JR Raphael, IDG

You can then create an entire collection of those single-tap time-savers for all the people you call or text often.

And with that, your phone’s communication capabilities have been upgraded considerably. A mobile phone that’s actually optimized for efficient calling and messaging — goodness gracious. What’ll they think of next?!

Get even more Android optimizing intelligence with my free Android Shortcut Supercourse. You’ll learn tons of time-saving tricks!

Why health might be Apple’s AI profit center

August 8, 2024 / Security – Computerworld

Like me, analysts expect Apple to eventually charge for access to some Apple Intelligence features, which is why I think the biggest opportunity for the company involves AI-augmented fitness and healthcare.

First, the thesis: Analysts Neil Shah (Counterpoint Research) and Ben Wood (CCS Insight) both told CNBC they believe Apple will eventually attempt to monetize Apple Intelligence, potentially as part of its Apple One bundle of services.

Shah points out that AI has the potential to become more personalized to users over time. This is particularly true for Apple, which has been designing Apple Intelligence from day one to prioritize privacy in all it does. The company has even begun making servers to drive some of its AI services, and as the services those servers provide become more sophisticated, it makes sense to charge for the more advanced tools it offers.

Privacy + AI = ?

The winning combination of Apple Intelligence along with the personal data gathered by the company’s devices and the continued research that makes it possible for AI to work with information without ever actually seeing the inherent data can’t be ignored. It’s that combination (along with the health sensors inside Apple Watch) that make AI-augmented digital health services ripe for monetization.

Think about the services Apple already offers that relate to health. Fitness+ might be the fee-based service, but it is supported by the Health app, years of in-depth research into wearable devices and health, and amazing technological manifestations to protect heart health, women’s health — even fall detection, crash detection, and Emergency SOS via Satellite. Many of these features already rely on various forms of machine intelligence, but generativeAI (genAI) can be far more creative in using the data points Apple’s devices collect — privately and only visible to you.

If Apple does choose to monetize these AI products, it won’t have had to look terribly far for inspiration. Palantir founder Peter Thiel sat on the same tech advice panel as Apple CEO Tim Cook, and the former company is working extensively in AI in healthcare, particularly (and contentiously) with the UK’s National Health Service (NHS).

The NHS is also working closely to develop ways to use AI to support its services. “We’re already seeing great applications of AI technology, and more work is under way to fully harness its benefits and use AI safely and ethically at scale,” the NHS said.

But while Palantir seems to need plenty of NHS data to run its operations, Apple’s approach seems to require less of that because Apple strives to ensure it sees as little as possible.

Would you pay for AI-augmented digital health?

Now, I don’t want to get into a relative conversation about the differences between Palantir and Apple’s approach to data privacy. I’m not a Palantir expert. But what does seem clear is that Apple’s billions of users might well be willing to pay for smart digital health services, and Apple Intelligence could deliver this.

Based on Apple’s direction of travel in digital health so far, these services would almost certainly focus on preventative health intelligence rather than actual remedies.

The intelligence would exist to provide early warning of symptoms, and to recommend what actions customers could take to handle a health crisis/intervention. Another manifestation (perhaps in conjunction with Apple’s satellite communications services) could be an international medical and emergency system for travellers.

Apple doesn’t need to base its AI monetization on health, of course. It has plenty of other strings for its bow. But one thing is certain — no company will spend tens of billions building cutting-edge services without attempting to recoup that investment down the line. With that in mind, I can’t believe Apple, which famously charges “reassuringly high” amounts even for things like spare iPhone charging leads, isn’t going to find some way to make money from Apple Intelligence.

Reading the room

Wherever it chooses to make that money back, I think it will look to where its core values around privacy, security, and intentionality in AI make the most difference. The company’s seemingly deliberate approach to introducing new features meets the public mood of a population that is becoming increasingly mistrustful of tech firms.

That means the solutions it has introduced so far provide value to most users while also protecting their privacy. Those services are almost certainly the thin end of a multi-billion dollar AI wedge, and Apple, more than most, understands the need to provide solutions that don’t scare customers. (Apple Pay on iPhone was also one of these, back in the day. Now it is the most dominant mobile payment system because Apple introduced it gently.)

This calm, measured intentionality will become increasingly visible over the coming two to three years as Apple puts the pieces in place to deliver fee-based Apple Intelligence services.

Fees aren’t the only way in which Apple will hope to profit from AI.

There are other options

Part of its payola should come in accelerating hardware sales as it ships the first end-to-end mobile-to-Mac ecosystem with AI inside and processors to power it up.

Apple will also be hopeful that its developer communities identify amazing new ways to deploy the large language models (LLMs) it has created within their apps. That’s great for developers, of course, but also gives Apple the competitive edge it needs to maintain hardware sales while also grabbing its slice of App Store revenue action.

But crucial to all of this is that Apple’s AI should be seen as a service, which is what Apple is going to try to build on once the initial Apple Intelligence release is done. That’s going to merit the introduction of additional premium AI tools and services aimed at what people are already doing with their devices.

These could include more advanced email analysis tools to help productivity professionals stay on top of increasingly demanding in-boxes. They might extend to premium automated Keynote presentation design and creation tools.

But the most likely space in which Apple will explore the opportunity for more sophisticated AI inside its devices will be around personal health, at the intersection of science, privacy, and digital arts.

Please follow me on Mastodon, or join me in the AppleHolic’s bar & grill and Apple Discussions groups on MeWe.

OpenAI releases new version of GPT-4o via Azure

August 8, 2024 / Security – Computerworld

ChatGPT-maker OpenAI has released a new version of its GPT-4o large language model (LLM), designed to simplify the process of generating “well-defined” and “structured” outputs from AI models.

“This feature is particularly valuable for developers who need to validate and format AI outputs into structures like JSON schemas. Developers often face challenges validating and formatting AI outputs into well-defined structures like JSON schemas,” the company wrote in a blog post, adding that an early release version of the LLM has been made available on Microsoft’s Azure OpenAI Service.

The structured outputs generated by the new version of the LLM, christened GPT-4o-2024-08-06, are a result of the LLM allowing developers to specify the desired output directly from the AI models.

JSON schema, according to OpenAI and Microsoft, is used by developers to maintain consistency across platforms, drive model-driven UI constraints, and automatically generate user interfaces.

“They are also essential for defining the structure and constraints of JSON documents, ensuring they follow specific formats with mandatory properties and value types. It enhances data understandability through semantic annotation and serves as a domain-specific language for optimized application requirements,” the companies explained.

Additionally, the companies said that these schemas also support automated testing, schema inferencing, and machine-readable web profiles, improving data interoperability.

The new LLM supports two kinds of structured outputs — user-defined JSON schema and strict mode or more accurate tool output.

While the user-defined outputs allow developers to specify the exact JSON Schema they want the AI to follow, the strict mode output, which is a limited version, lets developers define specific function signatures for tool use, the companies said.

The user-defined output is supported by GPT-4o-2024-08-06 and GPT-4o-mini-2024-07-18. Alternatively, the limited strict mode is supported by all models that support function calling, including GPT-3.5 Turbo, GPT-4, GPT-4 Turbo, and GPT-4o models.

GPT-4o was first announced in May 2024, as OpenAI’s new multimodal model, followed by GPT-4o mini in July 2024.

Microsoft has yet to make the pricing of the new model available on its pricing portal.

Critical time for the company and competition

The new model comes at a time when the company is undergoing critical changes in its leadership and is facing stiff competition from rivals such as Anthropic, Meta, Mistral, IBM, Google, and AWS.

This week, OpenAI co-founder John Schulman joined the list of departing OpenAI executives making the move to Anthropic — a rival LLM maker and provider that was founded by a group of executives leaving OpenAI.

Schulman follows Jan Leike, who made the move from OpenAI to Anthropic back in May.

Ilys Sutskever also left a senior role with OpenAI, but he is launching his own AI effort.

Another troubling indicator at OpenAI was the transfer of safety executive Aleksander Madry, who was mysteriously reassigned. This also follows OpenAI cofounder Andrej Karpathy’s departure back in February.

At the same time, OpenAI is also facing stiff competition for its LLMs. One such example is Meta’s newly unveiled Llama 3.1 family of large language models (LLMs), which includes a 405 billion parameter model as well as 70 billion parameter and 8 billion parameter variants.

Analysts and experts say that the openness and accuracy of the Llama 3.1 family of models pose an existential threat to providers of closed proprietary LLMs.

Meta in a blog post said that the larger 405B Llama 3.1 model outperformed models such as Nemotron-4 340B Instruct, GPT-4, and Claude 3.5 Sonnet in benchmark tests such as MMLU, MATH, GSM8K, and ARC Challenge.

Its performance was close to GPT-4o in these tests as well. For context, GPT-4o scored 88.7 in the MMLU benchmark and Llama 3.1 405B scored 88.6.

MMLU, MATH, GSM8K, and ARC Challenge are benchmarks that test LLMs in the areas of general intelligence, mathematics, and reasoning.

The smaller Llama 3.1 models of 8B and 70B, which have been updated with larger context windows and support for multiple languages, also performed better or close to proprietary LLMs in the same benchmark tests.

Another example is the release of Claude 3.5 Sonnet in June, which according to the LLM-maker, set new scores across industry benchmarks, such as graduate-level reasoning (GPQA), MMLU, and HumanEval — a test for coding proficiency.

While analysts and industry experts have been waiting for OpenAI to release GPT-5, company CEO Sam Altman’s post on X about summer gardens and strawberries has sparked speculations that the company is working on a next-generation AI model that can crawl the web to perform research.

Back to the future: Windows Update is now a trojan horse for hackers

August 8, 2024 / Security – Computerworld

A recent discovery has revealed a serious flaw in Microsoft’s Windows Update. Instead of protecting computers, it can be tricked into installing older, vulnerable operating system versions. This allows hackers to bypass security measures and attack computers even if they have the latest updates installed. It’s like dialing back time to find the perfect vulnerability to exploit.

Alon Leviev, a security researcher at SafeBreach, has unveiled a technique that lets malicious actors manipulate the Windows Update process to downgrade critical system components, rendering security patches useless.

Musk’s X under scrutiny in Europe for data privacy practices

August 8, 2024 / Security – Computerworld

Elon Musk’s X platform faces legal action in Ireland, with the Data Protection Commission (DPC) filing High Court proceedings over concerns related to the handling of European users’ personal data.

DPC has raised concerns about X’s use of public posts from the European Union and European Economic Area (EU/EEA) to train AI systems, including its chatbot “Grok,” according to an RTE report.

The commission alleges that X’s use of Grok violates GDPR guidelines on data protection and privacy.

DPC is also concerned about the planned August launch of a new version of Grok, allegedly trained on EU/EEA users’ personal data, which could exacerbate existing issues.

X has reportedly refused the DPC’s requests to halt data processing or delay the new Grok release.

The option to opt-out

DPC’s action seeks orders to suspend, restrict, or prohibit X from processing users’ personal data for developing, training, or refining machine learning, large language models, or other AI systems.

X’s global government affairs team has responded to the proposed order, saying that while many companies continue to scrape the web to train AI models with no regard for user privacy, X has done everything it can to give users more control over their data.

“Unlike the rest of the AI industry, we chose to provide a simple control to all X users, allowing them to decide if their public posts and engagement activity could be used to improve the models used by Grok,” the team said on the social media platform. “We also allow users to control all interactions with Grok, including deleting their conversation history.”

According to the RTE report, the DPC acknowledges that X has implemented some mitigation measures, such as an opt-out mechanism, but deems it insufficient.

The commission alleges that a significant number of X’s millions of European users have had their data processed—and continue to have it processed—without the benefit of these protective measures.

Worsening perception of X

This situation, coupled with the potential for hefty fines and ongoing regulatory scrutiny, contributes to a negative perception of X’s commitment to data privacy, according to Prabhjyot Kaur, senior analyst at Everest Group.

“The public and media scrutiny surrounding these issues can further damage X’s reputation, as users and stakeholders may view the company as irresponsible or non-compliant with fundamental privacy standards,” Kaur said. “Competitors that demonstrate stronger adherence to data privacy laws could attract users seeking more secure platforms, amplifying the reputational risk for X.”

The DPC also plans to refer the matter to the European Data Protection Board. Kaur noted that X, already with a history of GDPR non-compliance, now faces heightened pressure to improve data protection practices, enhance transparency, and restore user trust, particularly in Europe.

The DPC’s actions also underscore a broader trend of stringent data protection enforcement across the EU, signaling regulators’ readiness to take swift and decisive measures against perceived GDPR violations. This move sets a significant precedent for future cases.

“Meta, for instance, halted its AI training plans in June following GDPR complaints, demonstrating the regulatory pressure on tech companies to comply with data privacy laws,” Kaur said. “The involvement of the European Data Protection Board indicates that this issue may influence data protection policies and enforcement strategies across the EU.”

Musk’s X under scrutiny in Europe for data privacy practices

August 8, 2024 / Security – Computerworld

DPC has raised concerns about X’s use of public posts from the European Union and European Economic Area (EU/EEA) to train AI systems, including its chatbot “Grok,” according to an RTE report.

The commission alleges that X’s use of Grok violates GDPR guidelines on data protection and privacy.

DPC is also concerned about the planned August launch of a new version of Grok, allegedly trained on EU/EEA users’ personal data, which could exacerbate existing issues.

X has reportedly refused the DPC’s requests to halt data processing or delay the new Grok release.

The option to opt-out

According to the RTE report, the DPC acknowledges that X has implemented some mitigation measures, such as an opt-out mechanism, but deems it insufficient.

Worsening perception of X

How to train an AI-enabled workforce — and why you need to

August 8, 2024 / Security – Computerworld

Artificial intelligence (AI) is taking the business world by storm, with at least three in four organizations adopting the technology or piloting it to increase productivity.

Over the next two years, generative AI (genAI) will force organizations to address a myriad of fast-evolving issues, from data security to tech review boards, new services, and — most importantly — employee upskilling.

At the beginning of this year, 87% of 200 companies surveyed by Bain & Co., a management consulting firm, said they were already developing, piloting, or had deployed genAI in some capacity. Most of the early rollouts involved tools for software code development, customer service, marketing and sales, and product differentiation.

Companies are investing heavily in the technology — on average, about $5 million annually with an average of 100 employees dedicating at least some of their time to genAI. Among large companies, about 20% are investing up to $50 million per year, according to Bain & Co.

By 2027, genAI will represent 29% of organizational AI spending, according to IDC. In all, the AI market is projected to be valued at $407 billion by 2027.

“The way I often describe this is AI is sucking the air out of almost all non-AI investments in the whole tech world,” said Harvard Business School Professor David Yoffie.

As adoption soars, companies are struggling to hire people with much-needed AI knowledge and experience. Many are even cutting what they believe are unnecessary jobs and replacing them with AI-skilled workers, though the majority of organizations aren’t replacing employees, they’re retraining them.

A recent survey at business consultancy Heidrick & Struggles showed a 9% year-over-year rise in companies developing AI talent internally.

Bain & Co.

Not yet ready for adoption

Tech services provider AND Digital recently published its CEO Digital Divide report based on a survey in April among 500 UK respondents, 50 in the US and 50 in the Netherlands; it found 76% of respondents had already launched AI boot camps, even as 44% also said their staff remained unready for AI adoption.

“Our research indicates that fear of falling behind is a major driver behind this trend,” AND Digital said in the report. “Fifty-five percent of respondents told us that the pace at which new technologies are emerging is causing alarm among the senior leadership team.”

Emily Rose McRae, a Gartner senior director analyst, said 18 months ago she would have recommended organizations delay upskilling workers on genAI until they had specific use cases. Now, McRae sees workers using genAI behind the scenes — even at organizations that have banned it.

“If employees are using genAI despite bans, then organizations need to make sure their workforce understands the risks and realities of the tools,” McRae said. “A general genAI training should also include a clear discussion of potential risks, and why review of content produced by genAI is critical, even for interim internal content such as memo or email drafts.”

The poll by AND Digital also showed 62% of CEOs think moving too slowly to adopt AI is riskier than moving too fast. “Shockingly, 43% said they believe AI could replace the job of the CEO, underlining the widespread fears that the technology could destroy traditional job roles,” the report stated.

Experts don’t believe AI tools will replace workers; they envision workers who understand how to apply AI in their current positions will replace those who don’t. In fact, two-thirds of current occupations could be partially automated by AI, according to Goldman Sachs.

AI centric-training is needed

Building an AI team is an evolving process, just as genAI itself is steadily evolving — even week to week. “First, it’s crucial to understand what the organization wants to do with AI,” Corey Hynes, executive chair and founder at IT training company Skillable, said in an earlier interview with Computerworld.

“Second, there must be an appetite for innovation and dedication to it, and a strategy — don’t embark on AI efforts without due investment and thought. Once you understand the purpose and goal, then you look for the right team,” Hynes added.

Building an AI team is an evolving process, just as genAI itself is steadily evolving. Some of the top roles companies might need:

• A data scientist who can simplify and navigate complex datasets and whose insights provide useful information as AI models are built.

• An AI software engineer who often owns design, integration, and execution of machine-learning models into systems.

• A chief AI officer (CAIO) or leader to provide leadership and guidance in implementing and leading AI initiatives to ensure alignment and execution.

• An AI security officer to handle the unique challenges, such as ensuring adherence to regulations, data transparency, and internal vulnerabilities that come with AI models and algorithms, including adversarial attacks, model bias, and data poisoning.

• Prompt engineers who can craft and improve text queries or instructions (prompts) in large language models (LLMs) to get the best possible answers from genAI tools.

• Legal consultants to advise IT teams to ensure organizations abide by regulations and laws.

There are practical methods for building AI expertise in an organization, including training programs and career development initiatives to drive innovation and maintain a competitive edge.

Bain & Co.

Rethinking the workforce in terms of AI tools

“You’re seeing organizations thinking about their workforce in different ways than five or 10 years ago,” said Gustavo Alba, Global Managing Partner for Technology & Services Practice at Heidrick & Struggles, an international executive search and management consulting company.

Corporate AI initiatives, Alba said, are similar to the shift that took place when the internet or cloud computing took hold, and there was “a sudden upskilling” in the name of productivity. Major technology market shifts also affect how employees think about their careers.

“Am I getting the right development opportunities from my employer? Am I being upskilled?” Alba said. “How upfront are we about leveraging some of these innovations? Am I using a private LLM at my employer? If not, am I using some of the public tools, i.e. OpenAI and ChatGPT? How much on the cutting edge am I getting and how much are we innovating?”

Companies that aren’t upskilling their workforce not only face falling behind in productivity and innovation, they could lose employees. “I think you have a workforce looking at [genAI], and asking, ‘What is my organization doing and how forward looking are they?”

When it comes to upskilling workers for the AI revolution, Alba said, workers can fall into one of three basic archetypes:

AI technology creators, or those highly skilled employees who work at AI start-ups and stalwarts such as Amazon, OpenAI, Cohere, Google Cloud and Microsoft. They are a relatively small pool of employees.
Technologists who understand technology in general, i.e. CTOs, software developers, IT support specialists, network and cloud architects, and security managers. They may understand traditional technology, but not necessarily AI, and represent a larger pool of employees.
Workers who perform non-technical tasks but who can become more productive and efficient through the use of AI. For example, understanding how Zoom can use AI to summarize an online meeting into bullet points.

AI technology creators require no additional training. Technologists are relatively easy to train on AI, and they’ll catch onto its potential quickly (such as software developers using it to help generate new code). But the general workforce will require more care and handling to understand the full potential of AI.

Alba believes every employee should receive some training on AI, but the exact type depends on their job. The quality of work and the amount of work that can be accomplished by an employee using AI compared to one who doesn’t can provide a competitive advantage, he noted.

“When was the last time we said that about a technology: that an end-user goes off and figures out how to apply it to their workflow,” Alba said. “All of a sudden, we’re discovering on an individual basis how to use these tools, but imagine an organization being in the forefront and saying, ‘I can suddenly get three times more productivity and increase quality or an investment of X.’”

For example, a customer care organization could train its employees on how to use Microsoft Copilot, which can come as part of Office 365. Representatives can then feed customer queries into Copilot and obtain the most appropriate responses in seconds. Chatbots like Copilot can also generate high-level reports. For example, a user could produce a list of top customer issues over the past quarter to be used to improve service.

“You can imagine getting trained up around an agent on how and when to respond with intuitive messaging, especially if it comes to you in writing,” Alba said. “You can also imagine a world where a chatbot takes all those [past customer patterns and purchases] and then helps guide the conversation with them.”

GenAI training vs. real-world use

While many organizations offer some basic training on genAI, it’s not really doing much to increase adoption or deliver the productivity gains “that genAI was hyped to provide,” according to Gartner’s McRae. “This is particularly true if the organization has a generic genAI tool that is available to everyone in the organization, as opposed to tools embedded within role-specific software employees already use on a day-to-day basis.”

Training that’s not followed with on-the-job use is “wasted training,” McRae said, because workers still don’t have clarity on how it will apply in their workflow.

On-the-job application is particularly necessary because genAI use cases can be extremely broad and the learning curve too high for creating prompts that achieve usable results. It takes time and iteration, according to McRae.

“It’s not a fully generalizable skill — because each tool has different fine-tuning and on of a variety of LLMs underlying them, prompt engineering techniques that work with one tool may not work with another. [That] means a lot of iteration, even for experienced genAI prompt writers,” she said.

McCrae suggests companies include use cases and examples that not only demonstrate how the organization wants employees to use genAI, but also spells out the corporate philosophy about the technology. And it’s important to make clear what impact genAI will have on roles and workflows.

“For example, it could include sharing how genAI was used to develop first drafts of software programs, and software developers were assigned new workflows or roles that involved extensive review and testing of the GenAI-produced drafts,” McRae said. “Meanwhile, the organization was able to move people interested in software development — but without much experience in the field — into prompt engineering roles where they create the prompts to generate the first drafts of the software programs.”

Because of the risks associated with genAI hallucinations, and the fact that even if employees aren’t using genAI tools themselves they will be receiving content that has been produced using the technology, McRae recommends annual training on “Information Skepticism.”

“Information Skepticism training will be similar to how most organizations have an annual training on phishing followed by unannounced tests that direct any users who fail to remedial training, in order to manage information security risk,” she said.

Information Skepticism training should also be followed by followed by random tests to ensure employees are staying aware and vigilant of the risks, McRae said.

Heidrick & Struggles’s Alba believes the more hands-on employees get, the better — even beyond any bootcamps or courses academia can offer. He compared education on genAI to learning the possibilities of the Internet in the mid-1990s, saying it’s a new frontier and most organizations will have to “roll up their sleeves” and try out the tech to see where productivity gains lie.

“The reality is the academic part of it is interesting, but these are new models that we don’t even know what they can do yet,” Alba said. “I don’t think the educational institutions will catch up to the hands-on reality of playing around with these.”

Apple’s instructions to its new Siri GenAI offering illustrate the GenAI challenge

August 7, 2024 / Security – Computerworld

Deep within Apple’s systems is a variety of instructions it has given to its GenAI Apple Intelligence mechanism. The screen captures of those instructions provide a peek into Apple’s efforts to influence its GenAI deployment, and also illustrate the steep challenges in controlling an algorithm that is simply trying to guess answers.

The more explicit and contained an instruction, the easier it is for GenAI to understand and obey it. Therefore, some of the Apple instructions, such as “You prefer to use clauses instead of complete sentences”, and “Please keep your summary of the input within a 10-word limit”, should work well, AI specialists said.

But other, more interpretable commands from the Apple screen captures, such as “Do not hallucinate. Do not make up factual information,” may not be nearly as effective.

“I have not had good luck telling it not to hallucinate. It’s not clear to me that it knows when it is hallucinating and when it is not. This thing isn’t sentient,” said Michael Finley, CTO at AnswerRocket. “What does work is to ask it to reflect on its work, or to use a second prompt in a chain to check the results of the first one. Asking it to double check results is common. This has a verifiably good impact on results.”

Finley was also baffled at a comment that told the system to “only output valid JSON and nothing else.”

“I am surprised that they told it to only use valid JSON. The model is either going to use it or not,” Finley said, adding it has no practical or meaningful way to assess validity. “The whole thing is really unsophisticated. I was surprised that this is what is at the heart.” He concluded that “it was kind of cobbled together. That is not necessarily a bad thing.” By that he meant that Apple developers were under pressure to move the software out quickly.

The instructions under scrutiny were for new GenAI capabilities being built into Apple’s Siri. The dataset Apple will be using is far larger than earlier efforts, which is why it will only be available on the latest devices with the strongest CPU horsepower as well as the most RAM.

“Apple’s models for Siri have been small until now. Using GPT — arguably some of the largest models — means new capabilities,” Finley said. “As parameter counts get bigger, models learn to do things that are more indirect. Small models can’t role-play, larger models can. Small models don’t know about deception, larger models do.”

Clyde Williamson, product security architect at Protegrity, was amused by how the existence in a public forum of the comments, which were presumably not intended to be seen by Apple customers, nicely illustrates the overall privacy/data security challenges within GenAI.

“This does highlight, though, the idea of how security in AI becomes a bit fuzzy. Anything we tell an AI, it might tell someone else,” Williamson said. “I don’t see any evidence that Apple tried to secure this prompt template, but it’s reasonable to expect that they didn’t intend for end-users to see the prompts. Unfortunately, LLMs are not good at keeping secrets.”

Another AI specialist, Rasa CTO Alan Nichol, applauded many of the comments. “It was very pragmatic and simple,” Nichol said, but added that “a model can’t know when it’s wrong.”

“These models produce plausible texts that sometimes overlap with the truth. And sometimes, by sheer accident and coincidence, it is correct,” Nichol said. “If you think about how these models are trained, they are trying to please the end-user, they are trying to think of what the user wants.”

Nichol liked many of the comments, though, noting, “The instructions to keep everything short, I always use comments like that,” because otherwise, LLMs tend to be “incredibly verbose and fluffy.”

Macs are becoming more locked down

August 7, 2024 / Security – Computerworld

Enterprises are becoming increasingly impressed by the robust security of Macs, and Apple is locking its platform down even more firmly with macOS Sequoia and a couple of changes to improve defenses against malware and “camfecting.” This reflects the company’s continued mission to ensure platform security by design.

Gatekeeper empowerment

The first change is the biggest. Apple’s Gatekeeper protection is designed to stop people from running unsafe applications on their Macs. When you try to install software downloaded from the Internet, you are presented with a security warning before the application will work (though it has long been possible for Mac users to bypass the protection by Control-Clicking on the application icon).

Apple has abandoned this in the latest Sequoia beta. Now, users must actively open Settings > Privacy & Security to permit their system to run such apps on a per-app basis.

While the impact of this change is slight — you can still install and use apps obtained elsewhere — it should help prevent users from accidentally installing malware because it makes the whole process more intentional. Less-experienced users become less likely to be tricked into giving such approval by the app installation screen.

Apple recommends notarization

The real aim of the change is to prevent users who might be less tech-savvy from being tricked into bypassing Gatekeeper. In an ideal world, Apple would like all apps installed on Macs to at least notarized, the company confirms.

“If you distribute software outside of the Mac App Store, we recommend that you submit your software to be notarized,” Apple says. “The Apple notary service automatically scans your Developer ID-signed software and performs security checks. When your software is ready for distribution, it’s assigned a ticket to let Gatekeeper know it’s been notarized so customers can run it with confidence.”

This is a similar process to what Apple is trying to achieve on iOS devices in Europe. The goal is to secure the user and the platform, while also narrowing the size of the attack surface on its systems.

Camfecting and how to stop it

The second change will seem annoying to some, but does at least put Mac users in control. If you have ever installed screen recording or video conferencing software, you were probably asked to provide permission for those applications to capture your Mac screen. You likely went ahead and gave that permission and forgot about it — but that means applications you (or someone with access to your Mac) gave such permission to might be able to secretly continue recording your actions.

This improves in macOS Sequoia, which will require that you review and confirm this permission once a week. A dialog box will appear explaining the app wants to access the computer screen and audio, and giving you two choices: disable that permission, or “Continue to Allow” access.

While some might see this process as overly intrusive, it should help protect Macs against some in-person and malware-based camfecting attacks, as any application that has permission to access the camera/screen recording will be surfaced once a week. That means if an app you didn’t expect to see there appears on the list, you should take immediate steps to secure your device.

User controlled security

Seen in context, these latest security improvements mean the Mac is becoming better locked down as Apple works to make security protections you already have in place more understandable.

Take the Privacy & Security section of Settings for example: Over time, this has become an extensive, perhaps daunting, collection of options Apple has made easier to understand. In Sequoia, you can now more easily see how many apps enjoy full or partial access to the various settings and have a guide to help you manage those settings.

Again and again with its security improvements, Apple continues working to make security an intentional choice, explains what it is users are securing, and is creating device management APIs IT can use to ensure that their entire fleet remains as secure as it can possibly be — no kernel access required.

Please follow me on Mastodon, or join me in the AppleHolic’s bar & grill and Apple Discussions groups on MeWe.