Author: Security – Computerworld

Checkr ditches GPT-4 for a smaller genAI model, streamlines background checks

September 30, 2024 /

Checkr provides 1.5 million personnel background checks per month for thousands of businesses, a process that requires generative AI (genAI) and machine learning tools to sift through massive amounts of unstructured data.

The automation engine produces a report about each potential job prospect based on background information that can come from a number of sources, and it categorizes criminal or other issues described in the report.

Of Checkr’s unstructured data about 2% is considered “messy,” meaning the records can’t be easily processed with traditional machine learning automation software. So, like many organizations today, Checkr decided to try a genAI tool — in this case, OpenAI’s GPT-4 large language model (LLM).

GPT-4, however, only achieved an 88% accuracy rate on background checks, and on the messy data, that figure dropped to 82%. Those low percentages meant the records didn’t meet customer standards.

Checkr then added retrieval augmented generation (or RAG) to its LLM, which added more information to improve the accuracy. While that worked on the majority of records (with 96% accuracy rates), the numbers for more difficult data dropped even further, to just only 79%.

The other problem? Both the general purpose GPT-4 model and the one using RAG had slow response times: background checks took 15 and seven seconds, respectively.

So, Checkr’s machine learning team decided to go small and try out an open-source small language model (SLM). Vlad Bukhin, Checkr’s machine learning engineer, fine-tuned the SLM using data collected over years to teach what the company sought in employee background checks and verifications.

That move did the trick. The accuracy rate for the bulk of the data inched up to 97% — and for the messy data it jumped to 85%. Query response times also dropped to just half a second. Additionally, the cost to fine-tune an SLM based on Llama-3 with about 8 billion parameters was one-fifth of that for a 1.8 billion-parameter GPT-4 model.  

To tweak its SLM, CheckR turned to Predibase, a company that offers a cloud platform through which Checkr takes thousands of examples from past background checks and then connects that data to Predibase. From there, the Predibase UI made it as easy as just clicking a few buttons to fine-tune the Llama-3 SLM. After a few hours of work, Bukhin had a custom model built.

Predibase operates a platform that enables companies to fine-tune SLMs and deploy them as a cloud service for themselves or others. It works with all types of SLMs, ranging in size from 300 million to 72 billion parameters.

SLMs have gained traction quickly and some industry experts even believe they’re already becoming mainstream enterprise technology. Designed to perform well for simpler tasks, SLMs are more accessible and easier to use for organizations with limited resources; they’re more natively secure, because they exist in a fully self-manageable environment; they can be fine-tuned for particular domains and data security; and they’re cheaper to run than LLMs.

Computerworld spoke with Bukhin and Predibase CEO Dev Rishi about the project, and the process for creating a custom SLM. The following are excerpts from that interview.

When you talk about categories of data used to perform background checks, and what you were trying to automate, what does that mean? Bukhin: “There are many different types of categorizations that we they would do, but in this case [we] were trying to understand what civil or criminal charges were being described in reports. For example, ‘disorderly conduct.'”

What was the challenge in getting your data prepared for use by an LLM? Bukhin: “Obviously, LLMs have only been popular for the past couple of years. We’ve been annotating unstructured data long before LLMs. So, we didn’t need to do a lot of data cleaning for this project, though there could be in the future because we are generating lots of unstructured data that we haven’t cleaned yet, and now that may be possible.”

Why did your initial attempt with GPT-4 fail? You started using RAG on an OpenAI model. Why didn’t it work as well as you’d hoped? Bukhin: “We tried GPT-4 with and without RAG for this use case, and it worked decently well for the 98% of the easy cases, but struggled with the 2% of more complex cases., was something I’d tried to fine tune before. RAG would go through our current training [data] set and it would pick up 10 examples of similarly categorized categories of queries we wanted, but these 2% [of complex cases, messy data] don’t appear in our training set. So that sample that we’re giving to the LLM wasn’t as effective.”

What did you feel failed? Bukhin: “RAG is useful for other use cases. In machine learning, you’re typically solving for the 80% or 90% of the problem, and then the longtail you handle more carefully. In this case where we are classifying text with a supervised model, it was kind of the opposite. I was trying to handle the last 2% — the unknown part. Because of that, RAG isn’t as useful because you’re bringing up known knowledge while dealing with the unknown 2%.”

Dev: “We see RAG be helpful for injecting fresh context into a given task. What Vlad is talking about is minority classes; things where you’re looking for the LLM to pick up on very subtle differences — in this case the classification data for background checks. In those cases, we find what’s more effective is teaching the model by example, which is what fine-tuning will do over a number of examples.”

Can you explain how you’re hosting the LLM and the background records? Is this SaaS or are you running this in your own data center? Bukhin: “This is where it’s more useful to use a smaller model. I mentioned we’re only classifying 2% of the data, but because we have a fairly large data lake that still is quite a few requests per second. Because our costs scale with usage, you have to think about the system set-up different. With RAG, you would need to give the model a lot of context and input tokens, which results in a very expensive and high latency model. Whereas with fine-tuning, because the classification part is already fine-tuned, you just give it the input. The number of tokens you’re giving it and that it’s churning out is so small that it becomes much more efficient at scale”

“Now I just have one instance that’s running and it’s not even using the full instance.”

What do you mean by “the 2% messy data” and what do you see as the difference between RAG and fine tuning? Dev: “The 2% refers to the most complex classification cases they’re working on.

“They have all this unstructured, complex and messy data they have to process and classify to automate the million-plus background checks they do every month for customers. Two percent of those records can’t process with their traditional machine learning models very well. That’s why he brought in a language model.

“That’s where he first used GPT-4 and the RAG process to try to classify those records to automate background checks, but they didn’t get good accuracy, which means those background checks don’t meet the needs of their customers with optimal occuracy.”

Vlad: “To give you an idea of scale, we process 1.5 million background checks per month. That results in one complex charge annotation request every three seconds. Sometimes that goes to several requests per second. That would be really tough to handle if it was a single instance LLM because it would just queue. It would probably take several seconds if you were using RAG on an LLM. It would take several seconds to answer that.

“In this case because it’s a small language model and it uses fewer GPUs, and the latency is less [under .15 seconds], you can accomplish more on a smaller instance.”

Do you have multiple SLMs running multiple applications, or just one running them all? Vlad: Thanks to the Predibase platform, you can launch several use cases solutions onto one [SLM] GPU instance. Currently, we just have the one, but there are several problems we’re trying to solve that we would eventually add. In Predibase terms, it’s called an Adapter. We would add another adatpersolution to the same model for a different use case.

“So, for example, if you’ve deployed a small language model like a Llama-3 and then we have an adapter solution on it that responds to one type of requests, we might have another adatper solution on that same instance because there’s still capacity, and itthat solution can respond to a completely different type of requests using the same base model.

“Same [SLM] instance but a different parameterized set that’s responsible just for your solution.”

Dev: “This implementation we’ve open-sourced as well. So, for any technologist that’s interested in how it works, we have an open-source serving project called LoRAX. When you fine-tune a model… the way I think about it is RAG just injects some additional context when you make a request of the LLM, which is really good for Q&A-style use cases, such that it can get the freshest data. But it’s not good for specializing a model. That’s where fine tuning comes in, where you specialized it by giving it sets of specific examples. There are a few different techniques people use in fine-tuning models.

“The most common technique is called LoRA, or low-rank adaptation. You customize a small percentage of the overall parameters of the model. So, for example, Llama-3 has 8 billion parameters. With LoRA, you’re usually fine tuning maybe 1% of those parameters to make the entire model specialized for the task you want it to do. You can really shift the model to be able to the task you want it to do.

“What organizations have traditionally had to do is put every fine-tuned model on its own GPU. If you had three different fine-tuned models – even if 99% of those models were the same – every single one would need to be on its own server. This gets very expensive very quickly.”

One of the things we did with Predibase is have a single Llama 3 instance with 8 billion parameters and bring multiple fine-tuned Adapters towards it. We call this small percentage of customized model weights Adapters because they’re the small part of the overall model that have been adapted for a specific task.

Vlad hasd a use case up now, let’s call it Blue, running on Llama 3 with 8 billion parameters that does the background classification. But if he had another use case, for example to be able to extract out key information in those checks, he could serve that same Adapter on top of his existing deployment.

This is essentially a way of building multiple use cases to be cost effective using the same GPU and base model.

How many GPU’s is Checkr using to run its SLM? “Vlad’s running on a single A100 GPU today.

“What we see is when using a small model version, like sub 8 billion-parameter models, you can run the entire model with multiple use cases on a single GPU, running on the Predibase cloud offering, which is a distributed cloud.”

What were the major differences between the LLM and the SLM? Bukhin: “I don’t know that I would have been able to run a production instance for this problem using GPT. These big models are very costly, and there’s always a tradeoff between cost and scale.

“At scale, when there are a lot of requests coming in, it’s just a little bit costly to run them over GPT. I think using a RAG situation, it was going to cost me about $7,000 per month using GPT, $12,000 if we didn’t use RAG but just asked GPT-4 directly.

“With the SLM, it costs about $800 a month.”

What were the bigger hurdles in implementing the genAI technology? Bukhin: “I’d say there weren’t a lot of hurdles. The challenge was as Predibase and other new vendors were coming up, there were still a lot of documentation holes and SDK holes that needed to be fixed so you could just run it.

“It’s so new that metrics were showing up as they needed to. The UI features weren’t as valuable. Basically, you had to do more testing on your own side after the model was built. You know, just debugging it. And, when it came to putting it into production, there were a few SDK errors we had to solve.

“Fine tuning the model itself [on Predibase] was tremendously easy. Parameter tuning was easy so we was just need to pick the right model.

“I found that not all models solve the problems with the same accuracy. We optimized with to Llama-3, but we’re constantly trying different models to see if we can get better performance, and better convergence to our training set.”

Even with small, fine-tuned models, users report problems, such as errors and hallucinations. What did you experience those issues, and how did you address them? Bukhin: Definitely. It hallucinates constantly. Luckily, when the problem is classification, you have the 230 possible responses. Quite frequently, amazingly, it comes up with responses that are not in that set of 230 possible [trained] responses. That’s so easy for me to check and just disregard and then redo it.

“It’s simple programmatic logic. This isn’t part of the small language model. In this context, we’re solving a very narrow problem: here’s some text. Now, classify it.

“This isn’t the only thing happening to solve the entire problem. There’s a fallback mechanism that happens… so, there are more models you try out and that that’s not working you try deep learning and then an LLM. There’s a lot of logic surrounding LLMs. There is logic that can help as guardrails. It’s never just the model. There’s programmatic logic around it.

“So, we didn’t need to do a lot of data cleaning for this project, though there could be in the future because we are generating lots of unstructured data that we haven’t cleaned yet, and now that may be possible. The effort to clean most of the data is already complete. But we could enhance some of the cleaning with LLMs”

How much are companies willing to spend to get workers back to the office?

September 27, 2024 /

With more and more companies wanting to bring employees back to the office, I pointed out last week the ill-kept secret that there’s a widespread aversion to open office floor plans — or activity-based workplaces, as they have often evolved into today — and that it partially explains why many employees want to continue remote and hybrid work. 

This is not rocket science. For many years, it has been the consensus in the research community that open office landscapes are bad for both the work environment and employee performance. (There’s really no need for research at all — just talk to workers. They hate it.)

To be honest, open office environments are not downright bad. But it takes the right business, and the right type of people, for them to work. For example, I work in an industry where the ideal image is a teeming newsroom, where creative angles and news hooks are thrown back and forth, just as you see in a movie.

Even so, you don’t have to go back more than two or three decades to a time when most journalists, even in large newsrooms, had their own offices. That’s how Swedish offices used to look, people had their own rooms — not “cubicles,” but real rooms, with a door, and a small Do Not Disturb lamp. There was desk, pictures of the children (and maybe the dog), a plant and a small radio. It was a place where you could feel at home, even at work.

Then real estate development took over and today only 19% of office workers in Stockholm have their own space. The largest proportion, 42%, have no place of their own at all. And, according to researchers, it is the real estate companies that have been driving the transition to open office landscapes. 

It’s easy to see why: an open floor plan is, of course, much more surface-efficient than one with walls and corridors; it is much easier to scale up or down based on the tenants’ needs; and you can house more and larger companies in attractive locations in the city rather than large office complexes in the suburbs.

It’s not just the real estate industry’s fault. A little over 10 years ago, “activity-based offices” — otherwise known as hot-desking — arrived. Workers have neither their own room or desk. And here, the tech industry has taken the lead. 

When Microsoft rebuilt an office in Akalla in 2012, execs themselves called it one of the first large activity-based offices in Sweden, and it helped spark a trend where even the traditional companies and organizations adopted the “cool” scene from startup environments and Silicon Valley companies. (Puffs! European stools!) The office quickly evolved from cool to corporate.

Researchers actually welcomed the shift, as it at least gave people an opportunity to find a quieter place if they were disturbed or to avoid sitting next to colleagues they didn’t like. Then the COVID-19 pandemic hit and we know what happened next. Many people discovered how nice it is to work in their own room, at their own desk, that picture of the children, with maybe the dog at your feet, a plant nearby and some music. You didn’t need the Do Not Disturb light and there were no chattering colleagues.

As a Stockholm Chamber of Commerce’s survey found: 46% say that permanent workplaces in the office have become more important, and 45% of younger people would come in more if they had better opportunities for undisturbed work. (Whether it’s correlation or causality, I don’t think it’s a coincidence that the most important selling point for headphones these days is how good their noise canceling is. It makes public transportation bearable, certainly, and with headphones, you create your own room — even at work.)

As a result of these recent trends, property owners and companies alike find themselves in a tricky, but self-inflicted, position. To say the least, property owners have begun to see the disadvantages of the open solutions they pushed: vacancies in downtown office buildings are skyrocketing as tenants have reduced office space after the transition to hybrid work.

Yes, companies see the chance to save money by reducing office space, especially if employees aren’t there all the time anyway. But at the same time, they want their workers to be in the office more. And the employees say, “Okay, but then I would like to have my own place, preferably my own room.”

Of course, that equation doesn’t add up. And this is where the whole “return to office” trend is brought to a head. If company culture, creativity and productivity are so critical that employees need to be forced back into the office, how far are companies willing to go?

How big does the office space need to be, if everyone is to be there basically at the same time — if half also need their own desk to be productive, perhaps even a room of their own? 

Property owners and landlords would rejoice, but how many companies want to take on that cost? Very few, I would think.

Perhaps that tells us just how important a forced return to offices really is.

This column is taken from CS Veckobrev, a personal newsletter with reading tips, link tips and analyzes sent directly from Computerworld Sweden Editor-in-Chief Marcus Jerräng’s desk. Do you also want the newsletter on Fridays?  Sign up for a free subscription here.

HP’s new remote support service can even resurrect unbootable PCs

September 27, 2024 /

An unbootable PC is every remote worker’s nightmare. It usually means they need hands-on support that they’re not likely to find in their home office or neighborhood Starbucks.

Now there’s hope that even that catastrophe can be corrected remotely. At its Imagine event in Palo Alto, California on Tuesday, HP announced what it calls the industry’s first out-of-band diagnostics and remediation capability that will enable remote technicians to connect, diagnose, and fix problems, even if the PC won’t boot.

The service, launching Nov. 1, lets a technician, with permission from the user, connect to a virtual KVM (keyboard, video, mouse) under the BIOS/UEFI to run diagnostics and take remedial action. With the service, a tech could have corrected the CrowdStrike issue by replacing the flawed configuration file from the bad update, for example, and could even reimage the machine if necessary.

Marcos Razon, division president of lifecycle services and customer support at HP, said that the goal is to address 70%-80% of issues without requiring a stable operating system.

However, not all PCs will benefit, as the service relies on the Intel vPro chip more typically found in commercial PCs.

“Within the vPro chipset, you have a lightweight processor, a secondary processor that can access what in the past was called BIOS, but now it’s more UEFI,” Razon explained. “What this secondary processor allows us to do is to go under the BIOS before the booting process happens and take control of the machine.”

A security code must be accepted by the PC’s user before the technician can take control. “We don’t want anybody to access a PC without being able to secure that PC,” Razon said.

Constant virtual KVM

“The beauty of it is that we have a constant virtual KVM below the BIOS/UEFI,” he said.

The catch with existing remote-control programs is they need a PC that has successfully booted a stable operating system: “What happens is that if the PC has not booted completely, and the operating system is not running perfectly, you will not be able to take control of that PC,” he said.

Mahmoud Ramin, senior research analyst at Info-Tech Research Group, is impressed.

“Endpoint management tools usually fall short when a user faces serious problems with their hardware, such as boot failures and BIOS errors. Out-of-band technology can revolutionize remote endpoint management through bypassing operating systems and managing endpoints at the hardware level,” he said. “This innovation can help resolver groups seamlessly and immediately provide support to end users, reduce downtime by minimizing onsite visits, and enhance shift-left strategy through increased automation. HP’s out-of-band remediation capabilities can position it as a leader in remote endpoint support.”

The new service will be offered as an add-on to an HP Essential, Premium or Premium+ Support package with the purchase of any new vPro-enabled HP PC, the company said in a release. It will be extended to older HP commercial PCs in the coming months. It will initially be available in North America and the European Union, with rollout to other regions following. Razon said that the cost will be about US$12 per machine, per year, and HP is also working on a version for AMD processors, which it expects to release in the first half of 2025.

Microsoft 365 Copilot rollouts slowed by data security, ROI concerns

September 27, 2024 /

With the promise of greater office worker productivity, Microsoft 365 Copilot generated a lot of business interest even before it launched last November. (It was also for a while renamed Copilot for Microsoft 365; that changed earlier this month.)

That initial enthusiasm prompted many Microsoft 365 customers to put the generative AI (genAI) assistant into the hands of employees. But for many organizations, those rollouts have been largely limited to small trials at this stage. The reason — data security concerns and questions over the value the tool provides, said Max Goss, director analyst at Gartner. 

“Microsoft has done a very good job of generating interest; most organizations that we speak to now are trying it in some form or another,” Goss said. “What we are not seeing, though, is that those pilots translate into broader deployments.” 

A Gartner survey of 132 IT leaders at companies of a variety sizes in June — around half with 10,000 or more employees — showed that 60% of respondents have started pilot projects to deploy Microsoft 365 Copilot. But just 6% had finished their pilots at that point and were actively planning large-scale deployments. And only 1% had completed a Copilot deployment to all eligible office workers in their organization. 

Almost half — 48% of respondents — plan to move from their existing pilot projects to small-scale deployments later this year or in 2025.

“People want to use it — they’re excited about it — they’re just not quite sure how to use it, and that is hurting Microsoft from a deployment perspective,” said Goss, one of the report authors. (Full details of the survey methodology and findings can be found here.)

Others offer a similar perspective on early business uptake. “I would characterize it as ‘cautious optimism,’” said Dave Schubmehl, research vice president for IDC’s Conversational AI and Intelligent Knowledge Discovery research. “I think a lot of organizations are experimenting with Copilot.” 

Based on his own conversations with CIOs and IT leaders at organizations actively deploying M365 Copilot, Schubmehl gave a rough estimate of progress so far: around 20% are rolling out the AI assistant widely across their organization, another 20% are deploying it to select departments, and the remaining 60% are at a pilot or testing stage. 

Gartner survey on M365 Copilot

Where things stand on M365 Copilot deployments.

Gartner

Microsoft itself has talked up Copilot adoption without providing specific figures. 

During Microsoft’s Q4 FY24 earnings call, CEO Satya Nadella said that M365 Copilot adoption is growing fast, with a 60% increase in customers quarter over quarter. The number of customers more with than 10,000 seats more than doubled during this time, Nadella said. Microsoft has also highlighted some large customer deployments, noting Vodafone and Cognizant have purchased licenses for tens of thousands of employees.

Microsoft didn’t respond to a request for comment on the progress of M365 Copilot deployments and the number of business customers and individual users. 

Data security and governance

The ability for M365 Copilot to access organization’s data is one barrier to wider adoption.

Around 64% of respondents in the Gartner survey reported that information governance and security risks required significant time and resources to deal with, with concerns about data “oversharing” causing 40% to delay rollouts by three months or more. And 57% of respondents opted to manage risk levels by limiting their rollout to “low-risk or trusted” users.  

M365 Copilot’s large language models (LLMs) process responses based on an organization’s M365 data — all the files, emails, calendars and chat conversation data in applications and storage tools such as Outlook, Teams, Word, SharePoint and OneDrive. In theory, this shouldn’t be a problem: M356 Copilot follows customer’s existing user permissions and data security controls around documents stored in M365.

But if sensitive documents aren’t labelled correctly, they can be accessed by M365 Copilot when prompted to do so, with contents then surfaced in responses to users. 

This could allow payroll or company financial data to be exposed, to give some drastic examples. 

“This is, of course, not Copilot’s fault,” said Goss, “this is just the fact that you’re putting Copilot into an environment where permissions haven’t really been taken care of for years.”

The ability for M365 Copilot to surface sensitive data took has taken many businesses by surprise, said Brian Vecci, Field CTO at data protection software vendor Varonis.

The removal of a 300-seat minimum purchase requirement for M365 Copilot deployments in January encouraged a wider range of organizations to start pilots, but many were unprepared to deploy it more widely once the risks became apparent. “They’d give it to a few users in IT — 10 here, a dozen here, maybe a couple dozen users — and very quickly they realized there were significant privacy and security issues,” he said.

Managing which employees have access to certain files isn’t a new challenge for business, but the introduction of genAI assistants can substantially increase the risk of that sensitive data being exposed, either accidentally by an employee or by a malicious actor. 

“The better the information retrieval tool, the better your information governance has to be. And Copilot is, among many things, a very good information retrieval tool,” said Goss.

Business value remains hard to pin down

From an end user perspective, there’s plenty of interest in the tool. Almost all (98%) respondents in the Gartner survey said employees are excited to use it. Once granted a license, they want to keep it: 90% of respondents said employees would “fight to retain access.” 

But embedding the genAI assistant into employees’ workflow is a different story: 72% of respondents said employees struggle to integrate it into their daily routines, and 57% report that user engagement declines quickly after it’s implemented. 

At $30 per user each month for the enterprise version, price is a key factor in determining value, but it’s not the only one. Businesses also find that deploying M365 Copilot requires significant resource investment outside of licensing costs, such as change management efforts. Many respondents (73%) said that M365 Copilot deployments required higher change management efforts than expected, and 87% said end users require frequent engagement and education when the AI assistant is introduced. 

“One of our survey respondents put it perfectly: ‘The learning curve is longer than expected, particularly as the expectation is there’s no learning curve,’” said Goss. “We’re not talking about learning a new software or learning a UI, we’re talking about a new way of working. As a result of that, we are seeing change management being perhaps one of the biggest blockers.”

Combined, the governance and security challenges, user experience, and change management needs can make it difficult for some businesses to make the case for organization-wide M365 Copilot. Although the tool helped users save time, only 3% of respondents in the Gartner survey said M365 Copilot provides “significant” value at this stage. The majority said it is “somewhat valuable” with signs it will live up to expectations in future. 

It can be difficult to track the return on investment of genAI tools in the workplace, and M365 Copilot is no different. While there are examples where the value is clearer, said Schubmehl — an engineering services firm that finds it can reduce time engineers spend  with clients leading to lower project costs, for example — ROI is often more elusive for business. 

Then there’s the question of how employees should use the time they saved. “Should I use it to attend another meeting? Should I use it to go and take personal time and take my dog for a walk? It’s much more nuanced and that’s the challenge,” said Goss.

Of course, it’s still early days for the use of genAI tools in the workplace and for M365 Copilot as a product. Microsoft recently announced M365 Copilot “Wave 2,” with additional features in apps such as Excel, Outlook, and Teams, and, Pages, a new collaborative document that aims to bring the assistant into the workflow of teams and serve as a productivity tool. AI agents also offer the potential to expand what M365 Copilot can do in terms of automating business processes. And it’s likely there will be more updates to core features and underlying performance going forward. 

“We’re not talking about a product that’s more than a year old from a GA perspective, so I think it’s fair to say that you should expect some level of immaturity, and that will improve,” he said.  

However, Goss expects businesses will continue to have doubts about moving to wider M365 Copilot deployments, at least until the software matures to the point where value is no longer in question, and/or the cost model changes so organizations feel more comfortable tackling the challenges involved in deployments. 

“If those two things don’t change, and Copilot remains a similar product at a similar cost, then I think it could be a similar conversation in a year’s time,” he said.

IT gains open-source tools to manage iPhones, iPads, and Macs

September 27, 2024 /

If you happen to be, or know, an open-source advocate who’s fallen into Apple IT management, here’s some welcome news: all-remote, open-source device management platform Fleet has introduced iOS device support to supplement its existing Mac, Windows, and Linux device management capabilities.

This is, as regular readers will know I continue to insist, yet another sign of the growing maturity (and market share) of Apple’s products in the enterprise. The ecosystem that supports those products is growing to meet multiple business needs, and this now extends to enterprise admins who want open-source solutions to manage multiple platforms. That is what Fleet does.

Apple’s enterprise ecosystem gets a little open source

The deal is that Fleet’s GitOps-based approach to device management means IT departments can integrate existing GitHub repositories and workflows into their device management workflow. 

Who benefits most? IT admins managing multiple platforms. While needs and industry requirements differ, there are some who want the ability to manage multiple platforms from one place. Fleet gives them a tool to manage iOS (iPhone and iPad) devices, while also managing other platforms — all in the same Fleet instance.

“Our vision is to empower teams to manage their devices their way — without being locked into proprietary tools or vendor restrictions, Fleet CEO Mike McNeil said, banging the open-source drum.

I spoke with McNeil last year, when he explained that companies who choose tools on a per-platform basis can end up with problems handling it all.  “Juggling diverse platforms will often lead to adopting more vendors, which translates to higher costs and reduced efficiency due to the need for additional personnel with unique skill sets that translate poorly from one platform to the next,” he said then. “There’s no one-size-fits-all solution to this complex situation, no ‘easy button’ or single vendor that can solve these problems effortlessly.”

What about Fleet?

Fleet’s source code remains publicly available and contributable to by open-source developers. Fleet also offers some degree of scale. The company aspires to be as open in its business as in its code, with a highly detailed company handbook (which you can explore here), and claims to be attracting strong interest from fintech firms seeking to manage cross-platform fleets.

On its website, the company also explains: “This latest release is designed to simplify mobile device management (MDM) while giving your team the control and flexibility they need to scale effectively.”

What’s new?

The release supports automatic installation of software on hosts, self-service apps, and BYOD iPhones and iPads. Fleet’s support for Apple Business Manager (ABM) means admins can manage software installs, device settings, and compliance across employee-owned devices. And Volume Purchase Program (VPP) support makes it possible for admins to deploy corporate-purchased App Store apps across their devices. Fleet also supports automatic software installs to help maintain device and fleet compliance, though that feature isn’t yet iOS-compatible.

Fleet’s solution lets larger users add multiple ABM and VPP tokens across a single Fleet instance. While most smaller enterprises won’t put much store in that feature, larger businesses, especially Managed Service Providers (MSPs), will see immediate benefit from being able to use one instance of Fleet to handle multiple customer needs.

“Whether an MSP or an enterprise with multiple divisions, admins can set up separate workflows to automatically enroll devices and distribute apps through the App Store. This update simplifies the process of handling macOS, iOS, and iPadOS devices, providing a scalable solution for both MSPs and enterprises looking to centralize control while maintaining flexibility for different user groups,” the company says on its website.

What comes next?

Fleet has become one of the myriad of device management vendors locking in Apple support as they understand the critical nature of doing so. These vendors aren’t doing it for fun; they read the same market reports as everybody else, which of course means they also recognize that enterprise IT is changing. They know at their intuitive core that the future of business is as much about iPhones, iPads, and Macs as it once was about Roneo, Windows, and Letraset. It’s best to get to where that change is going, rather than be left behind.

Please follow me on LinkedIn, Mastodon, or join me in the AppleHolic’s bar & grill group on MeWe.

Intel and the US set to seal $8.5B CHIPS Act funding

September 27, 2024 /

Intel is on the verge of finalizing an $8.5 billion funding deal with the US government, under the CHIPS and Science Act, as part of an effort to boost domestic semiconductor manufacturing amid growing global competition, a Financial Times report said.

The deal, which has been in negotiation for months, would represent the largest subsidy package awarded to date under the act, which aims to reduce US reliance on foreign chip production and strengthen the country’s semiconductor supply chain.

According to the report, the deal could see a closure by the end of the year as the discussions were “at an advanced stage”. However, there “was no guarantee it will be finalized before the end of the year,” the report said citing two sources.

A request for comment from Intel remains unanswered.

In March 2024, the Biden Administration and Intel signed a preliminary memorandum of terms (PMT) for the $8.5 billion funding. This support package is part of Intel’s larger commitment to invest more than $100 billion in expanding its US manufacturing footprint, including building new chip plants in Arizona, Ohio, Oregon, and New Mexico.

The deal also comes with up to an additional $11bn in loans from the US government, likely to help Intel position itself as a critical player in the AI era of semiconductor technology.

The funding comes at a crucial time for Intel, which has been navigating financial struggles stemming from its massive multibillion-dollar investments in manufacturing expansion. With disappointing revenue in its PC and server chip business, Intel has cut costs and paused some projects, including a major initiative in Germany.

The company is also dealing with recent interest from Qualcomm and Apollo Global Management, which have reportedly explored potential investment opportunities in Intel.

Broader impact on the US semiconductor landscape

The funding is seen as the Biden administration’s strategic move to reduce dependence on foreign semiconductor supply chains.

“Intel holds strategic importance for the US as the only major US-headquartered semiconductor manufacturing company,” said Pareekh Jain, CEO of Pareekh Consulting. “Most other leading semiconductor manufacturers are based in Taiwan and South Korea. Maintaining semiconductor manufacturing within the US is vital for both national security and strategic interests.”

The US government’s support for Intel underscores the importance of maintaining American leadership in semiconductor technology.

Intel’s success is crucial not only for the company itself but for the broader US semiconductor ecosystem. With AI set to drive future demand for advanced chips, Intel’s manufacturing capacity and technological innovations will play a key role in maintaining US competitiveness in the global market.

“This agreement marks one of the largest investments ever in US semiconductor manufacturing, which will create over 30,000 good-paying jobs and ignite the next generation of innovation,” US Secretary of Commerce Gina Raimondo said in March.

The CHIPS Act has allocated $39 billion in direct government funding to bolster domestic manufacturing, with Intel receiving the largest individual award. Other recipients of CHIPS Act funding include Taiwan Semiconductor Manufacturing Company (TSMC) and Samsung, both of which are also expanding their US operations.

Intel’s ambitious plans and strategic challenges

While the deal offers a lifeline, Intel’s recent struggles have drawn scrutiny. Despite its ambitious plans, Intel has yet to fully capitalize on its investments. The company’s heavy spending on manufacturing has not been matched by sufficient revenue growth, particularly in its PC and server chip businesses. This has led to speculation that Intel could potentially sell parts of its business, including its manufacturing arm.

Rival chipmaker Qualcomm has explored buying a stake in Intel, while private equity firm Apollo Global Management has reportedly shown interest in a $5 billion investment. These overtures have raised concerns over potential antitrust issues, given that both Qualcomm and Intel operate in a highly consolidated industry.

The ongoing funding negotiations between Intel and the US government could be disrupted by any significant corporate restructuring or takeover. The CHIPS Act’s primary goal is to ensure that Intel remains a cornerstone of US semiconductor manufacturing, and the government’s backing is seen as a vote of confidence in Intel’s long-term strategy.

Therefore, it appears that it is in the US government’s best interest to support Intel through any financial difficulties to ensure continued investment in domestic semiconductor production, pointed out Jain.

“Even if Intel undergoes restructuring or is acquired, it is likely that only US-based firms will be considered as potential acquirers to safeguard national interests.”

What happens when everybody winds up wearing ‘AI body cams’?

September 27, 2024 /

Body-worn cameras, or “body cams,” are cameras attached to a person. 

Body cams have become ubiquitous in US law enforcement, with all police departments serving populations of more than 1 million implementing them by 2020. Nationwide, 79% of officers work in departments that use body cams. And in 2022, US President Joseph R. Biden Jr. signed an executive order in 2022 mandating them for federal officers in specific roles.

They’re so common now that it’s easy to forget how new they are. Police departments started testing them in earnest only around 2005.

Ten years ago, just a tiny minority of police wore body cams. But a series of high-profile incidences of abusive police officers in the mid-2010s triggered political pressure for police to use body cams. That’s why they are now standard equipment for police officers. 

Studies measuring their efficacy have been mixed. A 2014 pilot program in Orlando, FL, showed a 53% reduction in use-of-force incidents and a 65% drop in civilian complaints. But a large-scale study in Washington, D.C., in 2017 found no significant effects. 

Members of the public might believe that body cams exist to protect civilians from police abuse, and popular demand for cameras is almost entirely based on the belief that they’ll reduce overly aggressive police tactics. But, according to the Bureau of Justice Statistics (BJS), police and sheriff’s departments deploy body cams to improve officer safety, increase evidence quality, reduce civilian complaints, and reduce agency liability. 

They’re mainly seen as beneficial to the person wearing the camera (and the organization they work for). 

The spread of body cams to industry

Body cams are usually associated with cops. But the technology is increasingly deployed to prisons, private security, healthcare, education, retail, transportation, construction, highway construction and maintenance, sports, and just about any industry performing inspections.

These industries are finding that body cams improve documentation, increase safety, reduce theft and inventory loss, help with regulatory compliance, bolster employee accountability, generate evidence for use in lawsuits, and provide other benefits. 

The future of body cam tech is AI

The first body cams were primitive. They were enormous, had narrow, 68-degree fields of view, had only 16GB of internal storage, and had batteries that lasted only four hours.

Body cams now usually have high-resolution sensors, GPS, infrared for low-light conditions, and fast charging. They can be automatically activated through Bluetooth sensors, weapon release, or sirens. They use backend management systems to store, analyze, and share video footage. 

The state of the art — and the future of the category — is multimodal AI. 

A company called Polis Solutions partnered with Microsoft to develop an AI body cam system called TrustStat. Built on Microsoft’s Azure Government platform, TrustStat uses multimodal AI technology to study video, audio, and speech to interpret and analyze body language and actions, and other cues. According to the companies, it looks at entire interactions from start to finish to provide a nuanced understanding of police encounters with the public. 

It’s designed to solve the problem of sifting through thousands of hours of footage to extract actionable information, with vastly more advanced versions coming soon to body cams for police and across all sectors. 

AI ‘body cams’ for everybody

As the use of AI body cams grows to include all police departments, security personnel, and large numbers of employees across many industries, the public will also be getting AI body cams.

I’ve written in the past about the mainstreaming of AI glasses with cameras for multimodal AI. Remember Google’s Project Astra demo from Google I/O 2024? In that video, a researcher picked up a pair of AI glasses running Google Gemini and conversed with the AI about what they both were looking at. 

This is how multimodal AI glasses will work. 

Handling the video input could be similar to how Microsoft deals with captured screenshots for its Recall feature, available on Copilot+ PCs. In that system, Recall uses OCR to grab any text in the screenshots and convert it to ASCII. Recall then applies a CLIP-style embeddings model to the screenshot content. This creates vector representations of both textual and visual elements in the images, enabling semantic search. 

Using such a system in multimodal AI, a user could converse with their AI agent, asking questions about what the glasses were pointed at previously. 

These glasses will almost certainly have a dashcam-like feature where video is constantly recorded and deleted. Users can push a button to capture and store the past 30 seconds or 30 minutes of video and audio — basically creating an AI body cam worn on the face.

Smart glasses will be superior to body cams, and over time, AI body cams for police and other professionals will no doubt be replaced by AI camera glasses.

This raises the question: When everybody has AI body cams — specifically glasses with AI body cam functionality — nwhat does society then look like? 

Let’s start with the basics. Say, 10 years from now, when a police officer pulls over a driver, both the cop and the driver will record the encounter, save the video, and use AI to report back on what happened in the encounter on how each person interacted with the other.

This will likely prove popular with the public. When law enforcement controls the cameras exclusively, the occasional bad cop might be able to choose to record or not record, leading to a selective accounting of the incident. If people are also recording, that could improve the intent of body cams in the first place. Either way, everyone will be surveilled. 

Oracle founder Larry Ellison could have been referring to this earlier this month when he said, “We’re going to have supervision…. Every police officer is going to be supervised at all times, and if there’s a problem, AI will report that problem and report it to the appropriate person. Citizens will be on their best behavior because we are constantly recording and reporting everything that’s going on.”

All encounters between people could be subject to AI-analyzed body cam-like surveillance. 

A striking contrast exists between fictional predictions about mass surveillance and what actually happened. In George Orwell’s novel 1984, the government installed screens in every home, workplace, and public space to monitor citizens (and propagandize them). For a century, we’ve generally conceived of mass surveillance as something the government does to the public. 

As it happened, we, the citizens, installed cameras in those places — webcams, doorbell cams, security cams, and smartphones, recording, watching, and capturing video. AI camera glasses will simply add more cameras to the billions already in use, which will be more automated and actionable through AI analysis and interaction. 

Legally, footage from body cam-like AI camera glasses probably will andshould be considered “digital memory.” Of course, everyone has the right to natural memory of what they experienced. That right should be extended to digitally captured memory unless that “memory” violates another person’s privacy.

In free societies, AI camera glasses with AI body cam-like functionality won’t be banned. (If they’re required for a person to have clear eyesight, for example — because they’ll be fitted with prescription lenses — it would be legally difficult for the police to confiscate.)

The main point of all this is that we all know about police body cams. We should know now about AI processing of body cams. 

And it’s time to understand that the functionality of AI-based body cams is coming soon to everyone. 

Apple Intelligence in Europe? You might be waiting for a while

September 26, 2024 /

If you or your business happen to be nursing any hopes that Apple Intelligence will launch in Europe sooner rather than later, take a rain check on that AI expectation: Apple has failed to agree to a non-binding European Union pact to control artificial intelligence deployment.

Given the company is apparently in meetings with regulators with a view to clarifying how Europe’s DMA (Digital Markets Act) would be applied to its AI service, the fact it hasn’t (yet) signed on the dotted line suggests we’ll be waiting for it to reach Europe.

The EU AI pact has been inked by many other big tech firms; Adobe, Amazon, Google, Microsoft, OpenAI, Palantir, and Samsung are among 115 companies who’ve signed the document.

The EU AI Pact is a voluntary pledge to develop safe, trusted AI. “Artificial Intelligence (AI) is a transformative technology with numerous beneficial effects. Yet, its advancement brings also potential risks,” the pact explains. 

Neither Apple nor Meta have signed up

Interestingly, Apple isn’t alone — Meta hasn’t signed either. Nor have TikTok, Anthropic, or Mistral. 

Meta did, however, say it has not ruled out joining the pact down the road, according to Politico: “We also shouldn’t lose sight of AI’s huge potential to foster European innovation and enable competition, or else the EU will miss out on this one-in-a-generation opportunity,” Meta said.

The company has been critical of a lack of harmonization concerning AI implementation on a global scale.

What does the pact do?

The goal here is to put checks and balances in place around how AI is introduced in Europe. Signatories have agreed to comply with new European AI rules (the AI Act) which will be introduced in the coming years. 

The pledges include a commitment to:

  • Establish AI governance strategies to foster the uptake of AI in the organization and work towards future compliance with the AI Act.
  • High-risk AI systems mapping: Identifying AI systems likely to be categorized as high-risk under the AI Act. Those systems might be AI handling infrastructure, safety, or law enforcement, for example.
  • Promote AI literacy and awareness among staff, ensuring ethical and responsible AI development.

Additional pledges are voluntary, even within the agreement. These include a commitment to human oversight of AI and to ensure AI-generated content is clearly labeled as such. 

Where is Apple Intelligence?

Apple hasn’t said anything about all this, though it seems relevant to note that since it doesn’t yet offer Apple Intelligence in the EU, the company may not feel a need to do so. 

In the US, where Apple Intelligence is available, Apple in July agreed to a US presidential order governing AI technology. Elements of that agreement seem to echo those coming from the EU.

On reflection, it is possible that Apple’s acquiescence to the deal depends only on achieving a successful agreement regarding how the DMA will be applied to Apple Intelligence in the region. Apple CEO Tim Cook in August explained that the company is engaged with EU regulators, saying: “Our objective is to move as fast as we can, obviously, because our objective is always to get features out there for everyone. We have to understand the regulatory requirements before we can commit to doing that and commit a schedule to doing that, but we’re very constructively engaged with both.”

Don’t hold your breath?

While it is fair to note that the EU AI pact is different from the implementation of the EU DMA on Apple Intelligence, it’s hard not to think that Apple’s non-appearance as a signatory suggests more discussion is required before Apple launches its service in the region. If it even does.

Please follow me on LinkedIn, Mastodon, or join me in the AppleHolic’s bar & grill group on MeWe.

Amazon employees don’t want to be back in the office all week — survey

September 26, 2024 /

Amazon CEO Andy Jassy last week announced that the company’s more than 350,000 office workers will be required to work in the office five days a week by default. Jassy’s reasoning: ​​the move will result in better work.

But a new survey conducted by Blind, a forum for verified IT workers, asked 2,585 Amazon employees what they think about the change and that fully 91% don’t like the office requirement. And 73% will consider changing jobs because of the edict.

Blind writes that the change can particularly affect Amazon employees who are parents by removing flexibility and independence, as well as workers who were hired to work remotely or received adjustments for more flexible working hours.

The new arrangement is supposed to take effect on Jan. 2, 2025.