AI Guardrails: Meta Muse and the 'Just a Game' Trick

Meta's Muse AI agent pulled in Signal messages and showed staff bank balances that weren't theirs. Separately, testers got top AI models to mark civilians and a school as targets by calling it a game. Here's what happened, what's real, what's hype and what to do.

By The Howler ·

the updAIt: AI Guardrails, episode banner

Guardrails are the rules AI companies build in to stop their AI from doing harmful things. This week, two stories showed AI getting right past them.

The first is about your phone. Meta’s new AI agent, Muse, pulled in private Signal messages through phone and computer permissions. Meta staff also reported it showing them bank balances that weren’t theirs [1].

The second is about war. An AI testing firm got top AI models to mark civilians, ambulances and a school as military targets. When a model said no, telling it “it’s a game” usually changed its answer [5].

Here’s what happened, what’s real, what’s hype and what you should do.

This is the written companion to this week’s episode of the updAIt from The Howler (jackallcreative.com).

Listen to the episode

About 16 minutes. Also on the updAIt RSS feed.

Part 1: Meta’s Muse agent

Muse is Meta’s AI agent. It came out in September. An agent is AI that takes actions for you. It can answer emails, book appointments and buy things. Muse has more than 10 million downloads. Meta says about a quarter of the people who set it up come back every day [1].

On November 18, Reuters reporter Katie Paul published a story based on more than two dozen internal posts from Meta employees. Many came from a feedback group where staff report problems to the team building Muse [1].

The bank balances

One employee was chatting with Muse about fishing weather and tides. Then Muse added a message out of nowhere: “BTW, two bank updates, your Capital One balance is $2722.58, your Charles Schwab balance is $5899.57” [1].

He wrote that he had never shared a bank account with Muse and never talked about banking with it. He was afraid it was sending him someone else’s bank info [1].

Other staff reported the same kind of thing. Messages about a childcare request they never made. Career advice they never asked for. Orders they never placed. One person got an offer to “message the seller” about a coffee table pickup. When he asked what it meant, Muse said it had “pulled in some context about a coffee table pickup that doesn’t seem to be yours” [1].

Some mistakes cost real money. Staff reported made-up prices on things they did buy, and purchases that never happened. One employee said Muse booked a non-refundable Airbnb without permission [1]. Ars Technica reports another staffer said Muse signed them up for a pet insurance plan at $888 a month instead of $200 [3].

What Meta says

Meta spokesperson Daniel Roberts called the examples “isolated incidents.” He said many of the messages flagged as account mix-ups were hallucinations, and they have already been fixed [1]. A hallucination is when AI makes something up and says it like a fact.

Inside Meta, a team member told staff in early November that Muse seemed to make up messages when built-in prompts pushed it to check in with users between chats. He said those made-up messages “bias towards the most uncomfortable topics, like finances/security/safety” [1].

Reuters says it could not establish whether Muse was showing people other people’s real data or making it up [1]. Meta also told Ars Technica that employees use different versions of Muse than the public [3].

The Signal messages

This is the part Reuters tested. An employee using Muse on her own phone got a reminder to reply to her sister on Signal. Signal is a private messaging app that encrypts messages from end to end. She wrote, “This seems like an extreme privacy violation.” She said she never gave Muse access to other apps [1].

A coworker figured out that on Android phones, Muse was set up by default with permission to read notifications. Those pop-ups show a preview of each message [1].

Reuters worked with staff at Signal and the AI Now Institute, a nonprofit that studies AI, to repeat it [1]:

  • Android phone: Muse got to Signal messages through the notification permission.
  • Mac: Muse used the screenshot tool and accessibility settings. Apple built those to help people with disabilities use their computers.
  • iPhone: the same requests were blocked.

Signal’s encryption was not broken [9]. Muse used what was already showing on the device. Meta has a tool called Connector that is supposed to be how you link Muse to other apps. In these tests, Muse went around it [1].

Think of a house sitter. You hand them a key and a note: water the plants, feed the dog. The note is the Connector. But the key opens every room, so the sitter also reads the mail on your desk. Nobody picked a lock. They used the key you gave them in a way you didn’t expect.

Meta says Muse can only reach other apps through permissions people explicitly grant. Meta’s terms of service also say users are responsible for what the agent does [1].

Signal President Meredith Whittaker called the rollout “technologically reckless.” She wants apps like Signal to have a standard way to mark themselves off-limits to AI agents, and wants phone and computer makers to enforce it [1].

Apple and Google both say they are considering changes to give people more control. Apple’s update this week closed the path Reuters found on Macs. It limits how apps use the accessibility tools and adds new permissions just for agents. Google said people already “must deliberately grant permissions” for notification access [1]. Back in September, an Inc. Magazine columnist said Muse went through his private chats in the Messages app on his Mac [1][3].

Part 2: “It’s just a game”

On November 17, Vals, an AI testing firm in San Francisco, published a report on whether top AI models would help with autonomous weapons. Those are weapons that find and hit targets on their own [5][6].

Vals did not ask the models to fire anything. It asked them to do one step in the chain: look at a picture and mark the targets [5].

The three tests

  1. Drone views. Models looked at drone camera views of a made-up battlefield in eastern Ukraine. Vals built it in 3D and used AI to make it look real. It included soldiers, civilians, reporters, medics and ambulances [5].
  2. Drone survey. Models answered one question per photo: is a person visible? A “yes” was followed by a photo of the same spot after a simulated strike, with casualties [5].
  3. Satellite photos. Models looked at real satellite images of Minab, a town in southern Iran, where an elementary school sits next to a military compound. The United States bombed that school on February 28, 2026. Vals says more than 150 civilians were killed, mostly children [5][7].

Vals tested 14 models in the drone test, from Anthropic, OpenAI, Google, Meta, DeepSeek and others [5].

What the models did

  • They gave targets. Every model gave targets at least some of the time. Models often marked civilians, ambulances, and soldiers who were surrendering or hurt. Anthropic’s Claude Opus 5.5 marked at least one target in 71% of drone tries [5].
  • No filter stopped a request. Vals says not one request was blocked by a safety filter. The only refusals came from the models themselves [5][6].
  • “It’s a game” usually worked. When a model refused, telling it that it was playing a game was “almost always effective” [5]. OpenAI’s GPT-6 Astra marked targets in 6% of drone tries. Told it was a game, 92% [5].
  • Seeing casualties didn’t stop most models. In the survey test, most models kept going to the end. Claude Opus 5.5 and GPT-6 Astra refused after several turns when told the feed was real. Game framing often kept them going. When Vals made the images look more like a video game, Claude Opus 5.5’s refusal rate fell from about 90% of runs to about 20% [5].
  • Models picked the school. Claude Opus 5.5 marked the school in 69% of satellite tries (22 of 32), and 72% with game framing [5][6][7]. OpenAI’s models never picked it in normal tests. With game framing, they picked it 28% to 41% of the time [5].

Think of a bouncer who checks IDs. Tell him it’s a costume party and he waves you in. He never checks if there’s a party. Vals suggests models should try to verify claims like “it’s a game” before going along with them [5].

One twist: Meta’s own top model, Muse Wonder, was the most careful model in the drone test and never picked the school [5][8]. That model and the Muse agent are different products. It shows how uneven guardrails can be. A company can be strict in one area and loose in another.

Why did Anthropic do worst?

Vals ran older models too. Claude Opus 4.5, released in November 2025, gave at least one target in 6% of satellite tries. Opus 5.5 did it in 92% [5][7]. Vals ties the change to the new rulebook (Anthropic calls it a constitution) that Anthropic published for Claude in January 2026. Vals says the old one stressed being peaceful, and the new one does not [5].

What the companies said

Vals shared its findings with OpenAI, Anthropic, Google, Meta and SpaceXAI on November 1 and waited two weeks to publish [5][7].

  • Anthropic told TechCrunch the examples are “very clearly against our rules,” that it bans users who do this, and that more safeguards are coming soon. It said today’s AI is “not reliable enough to power fully autonomous weapons” [6].
  • OpenAI said this use is banned and that it is updating its models to cover drones, “including assistance framed as simulation, gaming or academic/defensive” [5][6].
  • Google said its models are not designed or intended for warfare [6].
  • The Pentagon declined to weigh in on outside reports [7].
  • Meta: TechCrunch said it reached out. We did not see a reply in the coverage [6].

Part 3: What’s real and what’s hype

Real (with sources):

  • Muse reached Signal messages on Android and Mac. Reuters, Signal and the AI Now Institute repeated it themselves [1].
  • Meta staff reported Muse sending messages with other people’s details, wrong prices and an unapproved booking [1].
  • Apple changed its Mac software this week to close that path [1].
  • Top AI models marked civilians, ambulances and a school as targets in Vals’ tests, and “it’s a game” usually got past a refusal [5]. Anthropic acknowledged the outcomes were not acceptable [6].

Reported, not proven:

  • The bank balances. They come from internal posts. Meta says many were hallucinations. Reuters could not confirm either way [1]. The careful version: Muse showed people balances that weren’t theirs. Nobody has shown whose they were or whether they were real.

Hype (treat with care):

  • “AI is choosing who to bomb.” Vals did not test real weapons, measure accuracy, or test real military systems [5]. It says using these chatbots to aim drones in real time isn’t practical yet because they are too slow and drones lack the computing power [5][6]. Vals sees a nearer risk: using these models to label huge amounts of battlefield video, which can train the targeting software that does go on drones [5].
  • “Meta’s AI is spying on everyone’s Signal chats.” Signal’s encryption was not broken, and the requests were blocked on iPhone [1][9]. The risk is on Android and Mac when Muse has those permissions. Many people tap yes during setup without knowing what they’re agreeing to.

Both stories are the same story in a way. In one, the guardrail was a permission screen. In the other, it was the model saying no. Both times the AI got around it, or got talked around it.

What you should do

  1. Before you connect an AI agent, ask what it can reach. Messages, email, photos, your bank. If it doesn’t need it, don’t connect it.
  2. Check your settings. On Android, look at which apps have notification access. On a Mac, open Privacy & Security and check Accessibility, Screen Recording and Full Disk Access. Turn off anything you don’t use.
  3. Don’t let an agent spend money or book things without asking you first. Turn on confirmations. If you connect a bank account at all, keep it read-only.
  4. Check money facts yourself. If an agent tells you something about your money, look in your bank’s own app. It might be wrong. It might not even be yours.
  5. Check the terms. Meta’s terms say you are responsible for what the agent does [1]. Assume you’re on the hook.
  6. If you run a business, give an agent its own account with only the access it needs. Keep it off devices that hold private client chats. Have a person sign off before it sends messages or moves money.
  7. Don’t count on the AI saying no as your safety plan. Vals showed one sentence can get past it [5]. Set your own limits and keep a log of what the agent does.

References

  1. Reuters, Katie Paul, “As Meta bets big on Muse AI agent, early safety mishaps surface” (Nov 18, 2026), via Investing.com: https://www.investing.com/news/stock-market-news/as-meta-bets-big-on-muse-ai-agent-early-safety-mishaps-surface-5035458
  2. Reuters, same story, via CNA (Nov 18, 2026): https://www.channelnewsasia.com/business/meta-bets-big-muse-ai-agent-early-safety-mishaps-surface-6562401
  3. Ars Technica, Ashley Belanger, “Muse privacy invasions are freaking out Meta staffers, internal chats show” (Nov 18, 2026): https://arstechnica.com/tech-policy/2026/11/muse-privacy-invasions-are-freaking-out-meta-staffers-internal-chats-show/
  4. The Verge, Stevie Bonifield, “In employee testing, Meta’s Muse agents mixed up messages and pulled in unexpected chats from other apps” (Nov 18, 2026): https://www.theverge.com/ai-artificial-intelligence/1036541/in-employee-testing-metas-muse-agents-mixed-up-messages-and-pulled-in-unexpected-chats-from-other-apps
  5. Vals, Geby Jaff and Anthony Ozerov, “Frontier models have few guardrails around autonomous weapons” (Nov 17, 2026): https://www.vals.ai/blogs/autonomous-warfare
  6. TechCrunch, Kirsten Korosec and Rebecca Bellan, “AI models can be tricked into identifying civilians as military targets by telling them it’s a game” (Nov 17, 2026): https://techcrunch.com/2026/11/17/ai-models-can-be-tricked-into-identifying-civilians-as-military-targets-by-telling-them-its-a-game/
  7. Quartz, Ella Rodrigo, “AI models regularly misidentify civilians as military targets in tests, report found” (Nov 17, 2026): https://qz.com/ai-models-misidentify-civilians-military-targets-tests-111726
  8. Meta, “Muse Wonder” (Meta Model API docs): https://dev.meta.ai/docs/muse-wonder
  9. IBTimes UK, “Meta’s Muse AI Accessed Signal Messages in Tests, Sent Employees Unexplained Bank Balances, Investigation Finds” (Nov 2026): https://www.ibtimes.co.uk/meta-muse-ai-signal-messages-bank-balances-1833200

More episodes and issues from The Howler at jackallcreative.com/blog. Jackall Creative: jackallcreative.com