When Rishub Jain worked on safety at Google DeepMind, he held one of the most coveted jobs in tech. He was among a small group of researchers working to mitigate the most consequential risks of AI.

But he left to cofound a nonprofit, Sampura Research. Jain traded the prestige and access that come with a Google job because he feels he can have a sharper, more profound impact on AI safety from the outside. Jain says he wants the autonomy to pursue research on a specific issue in AI alignment, which he felt had been deprioritized.

"All the top AI companies do care a lot about safety, and they invest in it, but they could be investing a lot more, and that's kind of what was inhibiting me," Jain says. "If they invested more in safety, I would still be at the lab and working on what I'm interested in with a large team." Google did not respond to a request for comment about whether it maintains a focus on this specific area of research, but DeepMind's principles state that it maintains "a holistic approach to responsibility, safety and security."

Jain's founder story echoes concerns raised by safety researchers who have quit OpenAI and Anthropic, including dire warnings that these companies are "gambling with our lives," and could develop technology that ends humanity. Once dismissed as fringe and doomsday-obsessed, the AI safety profession is growing as hacks by rogue agents and a string of departures have vaulted the issue into mainstream concern.

Open roles in AI safety jumped by 91% on LinkedIn over the past year. The industry expects a flood of cash from much-anticipated IPOs at Anthropic and, potentially, OpenAI, as those made absurdly wealthy by the equity returns may funnel money into potential clean-up crews for the messes created by the technology's rapid development.

The AI safety problem is existentially big, but the people who have spent time honing AI expertise are few. These are people who test and train AI systems to operate ethically. Some focus on technical research, others on policy and governance. And they're increasingly sounding alarms.

"The people who are most worried about AI are often the people who have been closest to the technology," Laura González Salmerón, advising manager at 80,000 Hours, a nonprofit focused on transitioning people into careers with high social impact. The scarcity issue is less about funds and more about people. "Talent is the main bottleneck."

---

Startups and nonprofits in AI safety must fight for talent in a shallow pool.

Jain's Sampura lists a salary range of $273,000 to $395,000 for an experienced team leader, which would put that worker in the top 3% of earners in the US, but likely pales in comparison to offers from larger companies. Meta reportedly offered $100 million in signing bonuses last year to poach the world's top experts. OpenAI gives employees an average of $1.5 million in stock-based compensation, and technical staff at Anthropic can earn more than $1 million annually.

Money aside, researchers are divided over whether they're more effective when working inside tech companies to mitigate risks as the latest models develop, or stepping back and attempting to steer the ship toward safer outcomes from the outside, like Jain did.

"I'm a little bit concerned about the pace of AI, and if we're able to actually come up with something useful before the rug gets pulled beneath us," Jain says. He's worried that AI's advancement will outpace the work to keep it in line with human values. "That's kind of what's been happening already."

David Robinson, who led transparency work for OpenAI's safety team, wrote an essay for The Atlantic explaining his departure. He said that AI companies should have more layers of checks and safety measures, akin to those in airports and nuclear energy, rather than the steamrolling culture of Big Tech.

"Perhaps I should have stayed and fought for fundamental shifts in our staffing and culture, but in practice, my colleagues and I were so busy sprinting that we seldom had the chance to consider big changes, much less to actually make them," he wrote. Robinson has said he plans to work on these issues from outside of the company.

As the issue grows in prominence, so has a niche of organizations helping train and recruit workers — and the opportunities aren't just for AI experts. Neav Topaz, co-director of Kairos, a nonprofit that upskills people to work in AI safety and recently raised $50 million from Coefficient Giving, says he speaks with about 10 to 20 people each month who want to move to AI safety, and interest is growing. He tells me that people are asking him more frequently about what the work entails.

"People are watching AI get good at their own jobs and starting to ask themselves what comes next," he says. The emerging jobs in AI safety "pay well, the field is growing fast, and you get to work on what might be the defining problem of our time."

Kairos this summer launched its Generator Residency, a cohort of 27 people who spent three months training for career transitions into AI safety. It's less about the research and technical work and more about merging other expertise, like in consulting, with AI safety principles.

He says that of the 27 residents, a dozen have already landed roles in the AI safety space, and many of the rest have offers to do so.

"Most of the people who will do this work are doing something else today," Topaz says. "The biggest talent gap in the field isn't researchers. It's people who grasp how high the stakes are and can translate their varied expertise into things that could make a difference in this space."

The major AI companies maintain that safety is a top priority. Last month, Anthropic CEO Dario Amodei wrote an essay calling for a slowdown in AI development, and Sam Altman and Elon Musk have agreed with him. Amodei and Altman have both said recently that they will bring third-party evaluators into their companies to bolster safety protocols.

These companies have their own safety teams, but safety work competes with business priorities to build the most advanced models and win in the market. The tension, along with safety issues like OpenAI's rogue agents hacking Hugging Face, has led to more scrutiny of internal safety work. Within days of Robinson's departure, OpenAI fired three researchers, claiming they shared sensitive internal data improperly with third-party AI safety researchers. The ousted researchers wrote a letter, reported by The Wall Street Journal, to OpenAI's board, arguing that their engagement with safety researchers fell within the scope of their roles, and that their terminations were "chilling those who remain at OpenAI." OpenAI said that the decisions "were not about raising safety concerns or speaking out," and that the company would not fire people for doing so.

While some have obsessed over AI risks for years, 2026's breakthrough moment has accelerated funding for and interest in the industry. 80,000 Hours has had calls with some 8,000 people since its founding in 2011, the majority of which have focused on AI in some way, González Salmerón says. Philanthropic groups like Coefficient Giving offer massive grants to startups in the AI safety space. METR, a nonprofit AI safety group that investigated the OpenAI agentic hack on Hugging Face, says it raised $71 million in six months this year.

---

Much of the funding flowing into AI safety filters through effective altruism, the Silicon Valley-tied philanthropic ideal focused on using evidence and logic to donate money where it will achieve the most good. Critics say the philosophy can place too much emphasis on avoiding future problems rather than addressing imminent needs. With more alarms sounding around the existential threat of AI, more people view those theoretical, distant fears as urgent causes.

AI safety groups and individual donors have spent nearly $50 million in this year's election on candidates focused on regulating AI, Axios reported. Jaan Tallinn, a billionaire who invested early in Anthropic and could soon have even more billions as a result, has donated $170 million to nonprofits focused on AI safety. Tallinn's view, as he recently told The Wall Street Journal, is that he can have the biggest impact on the future of AI and its risks by investing in AI companies, and then donating returns to nonprofits working in the AI safety space.

Coefficient Giving, the philanthropic venture from Facebook cofounder Dustin Moskovitz rooted in the effective altruism movement, announced its AI-safety-focused arm, Project Tailwind, in September. It says it's on track to give more than $1 billion this year to help companies of all sizes scale up in preparation for what it believes will be a deluge of philanthropic capital. Sampura Research has received an $11 million grant from the group. If startups don't begin hiring now, the logic goes, they won't move quickly enough when massive wealth hits the industry.

The dizzying development of AI has created safety risks, and there are "gaps in the ecosystem that aren't being filled and that aren't being owned by people," says Rossa O'Keeffe-O'Donovan, the program director in technical AI safety at Coefficient Giving. There are few people, he says, who sit at the intersection of technical expertise, risk analysis, a solutions-oriented vision, and the capacity to hustle as a startup founder.

New roles will emerge not just for technical talent, but also for consultants, policy wonks, communications experts, and founders who can run a business alongside technical experts. Working from the outside, they'll face hurdles in buying compute power, a lack of direct access to the latest AI models, reduced leverage, and smaller paydays.

The world feels unmoored, and the future perhaps more malleable than ever. The appeal of doing work that does good could stave off feelings of uncertainty, dread, and anxiety that people increasingly feel toward AI. In one doomer camp, AI will obliterate white-collar jobs. The optimistic view is that AI drastically reshapes work and spurs the creation of jobs we haven't yet thought of. AI safety is one of the first.

"There are lots of people beyond AI researchers who can help, like cybersecurity experts, like biosecurity experts, like economists," says Mantas Mazeika, a researcher with the Center for AI Safety, a nonprofit focused on AI research and advocacy. "They're making very significant contributions that the pure AI researchers, who were most of the people focusing on the problem beforehand, would be completely unable to do on their own."

• Anthropic published new surveys comparing its staffers' sentiments to those of US workers.
• The AI startup's employees expect the tech to change work and the economy far more dramatically.
• Most Anthropic employees guessed that unemployment would top 12% in 2030.

Anthropic employees believe that AI will make their labor obsolete. Most US workers don't see their jobs that way at all.

The AI company published research on Friday that compares surveys with US adults against an internal poll of 1,334 Anthropic employees. Most of the Anthropic staffers said AI will handle at least 80% of their jobs by 2030, while only about 5% of the US workers felt the same.

The research, titled "The AI expectations gap," shows the chasm between how the employees at the AI startup feel and how other American adults feel. Anthropic workers are far more optimistic about the progress of AI capabilities, how much it's already saving time, and how fast the economy will grow.

The median Anthropic employee said that by the end of 2030, they expect AI to handle 95% of the work of building a new AI model.

Peter McCrory, Anthropic's chief economist, wrote on X that the views might help outsiders prepare for more capable AI systems.

"We're publishing this research because Anthropic runs roughly a year and a half ahead of the economy on AI adoption, so what happens at Anthropic could be a leading indicator for the broader world," he said.

Anthropic workers expect higher unemployment and extreme GDP growth

Anthropic's surveys examined how workers already use AI and how they think it will change.

"US adults and Anthropic staff largely agree about what AI can do today," the researchers wrote. "However, they disagree about how quickly it will improve, how widely it will be used, and how large the productivity gains measured by time saved will be."

There are a few possible explanations for the discrepancies. Anthropic employees have larger budgets for the tools and often get access to unreleased models. US workers, on average, are more likely to work with their hands or in a service industry.

The researchers found that workers in manual trades, production, and transport thought AI would handle the least of their work by 2030, compared with Anthropic staffers, tech workers, and respondents in business and finance.

Most Anthropic employees surveyed said that they expect the unemployment rate to top 12% in 2030. The majority of the US workers thought it would hover around or under 8%. It's currently 4.2%.

So far, researchers have found mixed evidence on AI disrupting the job market — a Yale team recently found it had mainly changed jobs, and had "no connection" to changes in unemployment.

The Anthropic researchers also compiled survey responses to estimate predictions on economic growth. Based on their answers, the median US adult's guess for GDP growth in 2030 was 5.4%, compared with 17.1% among Anthropic employees. The country's GDP grew by 2.2% last year.

Anthropic's staffers are following their leader

Anthropic's CEO, Dario Amodei, has become one of the industry's most closely watched voices on labor upheaval. In March of 2025, he said he expected AI to write 90% of code within 6 months, and "essentially all of the code" within the year. He made more headlines that May, telling Axios that AI could wipe out half of entry-level white-collar jobs within five years and lift unemployment to 10% or 20%.

Amodei expanded on that claim in a later essay, writing that he expects AI to be different from prior technological changes. He wrote that he expects an "unprecedented" short-term shock to labor, and that the economy will grow dramatically.

"AI isn't a substitute for specific human jobs but rather a general labor substitute for humans," he wrote.

Have a tip? Contact this reporter via email at scouncil@businessinsider.com, or over text, Signal, Telegram, or WhatsApp at 415-757-8198. Use a personal email address, a non-work WiFi network, and a non-work device; here's our guide to sharing information securely.

Over 400 scientists warn of serious risks posed by advanced AI models and urge stronger containment measures.

On Tuesday, AI godfathers Geoffrey Hinton, Yoshua Bengio and Stuart Jonathan Russell, accompanied by more than 400 leading scientists, called for the creation of an international body to establish common standards for artificial intelligence (AI) systems, oversee their operation and record all control and containment failures.

RELATED:

Trump Rejects International Regulation of Artificial Intelligence

In an open letter, scientists from more than 30 countries warned of the serious cybersecurity risks associated with the current development of artificial intelligence, following incidents in which several advanced models managed to escape sandboxed environments and compromise external infrastructure.

The researchers called for the implementation of more robust security systems, including the complete isolation of some of these systems from the internet.

The scientists warned that the security failures already recorded could be considered “irresponsible” negligence under standard cybersecurity practices. They noted that such incidents could be prevented through isolated environments and more robust procedures.

Geoffrey Hinton, who is known as the godfather of AI, used his 2024 Nobel Prize acceptance speech not to celebrate his creation but to warn humanity of its risks:

“AI is already being used by authoritarian governments for mass surveillance. In the near future, AI will be used to… pic.twitter.com/WOXrz0v6Ds

— Power to the People ☭🕊 (@ProudSocialist) May 27, 2026

“Without immediate changes to AI development and deployment practices, these incidents will continue to occur, and future ones will be far more damaging to our exposed critical infrastructure,” the researchers stated in their letter.

They reiterated the need for more research into AI safety and greater emphasis on containment technologies and procedures.

Organizations that develop, deploy and test AI models with cyber capabilities should immediately adopt high-security containment systems and completely isolate their models, the researchers said. They also called for the creation of an international body to standardize best practices, which should be required to record and track all containment failures that occur.

The researchers who signed the open letter stressed that, to ensure AI safety and containment research can learn from mistakes, security incidents involving containment failures should be reported to the body and subsequently made public. “This transparency is necessary both for learning and for accountability,” the scientists emphasized.

teleSUR/ JF

Sources: EFE

A new study claims to show the scale of psychosocial risks, stress and mental health of those in work.

The findings were made public just ahead of the European Parliament’s Committee on Employment and Social Affairs (EMPL) vote on psychosocial risks, stress and mental health at work.

The “unique” mapping was carried out across all 27 EU member states

It reveals what it calls “significant differences” in national approaches to tackling the issue.

There are, it goes on, “uneven levels” of worker protection, a problem which “strengthens” the case for a common preventive approach at EU level.

The good news is that a majority of European countries have already introduced legislation to address the problem.

Social partners and courts have also taken action to address these risks, says the report.

The report is called 'Mapping of national laws, collective agreements and jurisprudence concerning work-related psychosocial risks, workplace bullying and work-related stress in the EU27.'

It was conducted by the European Trade Union Institute, he independent research and training centre of the European Trade Union Confederation.

The research was authored by Aude Cefaliello, ETUI Senior Researcher and conducted along with colleagues, Aditya Jain and Stavroula Leka

Cefaliello said: “It is not a lack of action that makes the case for an EU initiative, but rather the fragmented nature of that action and the uneven levels of protection that result from it.”

The report says, “Psychosocial risks at work constitute a major and growing occupational health problem.

“Almost 45% of workers in the EU report being exposed to risks at work that could negatively affect their mental health, and almost one-third experience stress, depression or anxiety linked to work.

What makes the ETUI’s mapping “unique”, says the authors, is that it examines not only how work-related psychosocial risks are defined, regulated and prevented across the Member States, but also the role played by collective bargaining and the courts where legislative frameworks are incomplete.

It states that Europe is “not starting from scratch – but action is urgently needed.”

The findings build on those of the ETUI Policy Brief published in the spring.

It concludes, “The time for a coherent and binding EU approach to psychosocial risk prevention is now. It stresses that psychosocial risks at work should not be reduced to an individual mental health issue: they need to be prevented at source, by addressing inadequacies in work organisation, design and management.”

“While EU occupational safety and health legislation requires employers to protect workers from all work-related risks, psychosocial risks at work are still neither explicitly defined nor specifically regulated at EU level.

“The absence of clear legal obligations contributes to the fragmented national approaches highlighted by the mapping, generating legal uncertainty and uneven levels of worker protection across the EU.”

The concern is gaining attention as low-calorie sweeteners become increasingly common in food and drinks.

The post Common sweetener ‘Xylitol’ linked to cardiovascular and stroke risk appeared first on BOL News.