• Human in the Loop
  • Posts
  • #36 Edition: Can AI firms set limits on how and where the military uses their models?

#36 Edition: Can AI firms set limits on how and where the military uses their models?

PLUS: Block cut 4,000+ roles while framing it as an AI operating shift

Hey, it’s Andreas.
This newsletter is now read by 30,000+ people. Thank you - especially to everyone who shared an edition or forwarded the subscribe link. That’s been one of the main growth engines

If you like what you read, share it with one colleague or friend: Subscribe.

In today’s issue:

  • Anthropic vs the Pentagon escalated over one clause: “any lawful use”

  • Block cut 4,000+ roles while framing it as an AI operating shift

  • ETH Zurich on AGENTS.md and why more instructions can make agents worse

  • And much more.

Let’s get into it.

Weekly Field Notes

🧰 Industry Updates

🌀 Anthropic updates Claude Cowork, new enterprise plugins, and Claude Code remote control → Cowork now supports scheduled tasks and org-shared plugins, while Claude Code sessions can be remotely resumed from mobile/web on the original machine (inspired by OpenClaw).

🌀 Anthropic accuses DeepSeek, Moonshot AI, and MiniMax of “industrial-scale” Claude distillation → Anthropic says the firms used ~24,000 fraudulent accounts to generate 16M+ exchanges with Claude to extract capabilities via distillation.

🌀 Block (Square + Cash App) on AI-driven restructuring and layoffs
→ Block announced ~4,000 job cuts (about 40% of staff) right after reporting strong results, framing it as an “AI + smaller teams” operating shift.

🌀 Google launches Nano Banana 2 (Gemini Flash Image) → Pro-level image quality at Flash speed, plus Google Search grounding for accurate real-world renders and better text + character consistency.

🌀 OpenAI partners with four consulting firms - BCG, McKinsey, Accenture, and Capgemini - in the so-called Frontier Alliance → The goal is to help deploy AI coworkers faster across enterprises.

🌀 QuiverAI on Arrow 1.0 (SVG-native model) public beta → QuiverAI came out of stealth with Arrow 1.0 in public beta - and it hit #1 on Design Arena’s SVG Arena leaderboard within 24 hours (Elo 1583).

🌀 Inception unveils Mercury 2, a diffusion-based reasoning model built for real-time → Claims ~1,000 tokens/sec output and ~5x faster throughput than speed-optimized LLMs.

🌀 Notion launches Custom Agents (autonomous AI teammates) → Automates recurring workflows across Slack, Mail, Calendar, Figma, and Linear.

🌀 Perplexity launches Computer, a long-running cloud agent → Autonomous workflow engine that can run tasks for hours or even months.

🌀 Nous Research releases Hermes Agent → A persistent, local-first personal agent you run on your own machine, designed to automate recurring personal workflows with memory + tool access.

🎓 Learning & Upskilling

📘 Philip Kiely (BasetenAI) on Inference Engineering (free book) → Free e-book about the inference stack (architecture, hardware, software, techniques and production).

📘 Anyscale launched a free “Introduction to Ray” course → A free, self-paced Ray 101 path covering Ray Core, Data, Train, Tune, and Serve.

📘 Anthropic with new Claude course catalog (Skilljar) → A full set of free courses (Claude 101, Claude Code, Claude API, MCP, agent skills, AI Fluency etc.).

📘 Anthropic stack overview: Claude AI vs Claude Code vs Claude Cowork
→ Simple decision rule: use Claude AI for thinking and writing, Claude Code when the work lives in a repo, and Cowork for cross-app file workflows.

🌱 Perspectives & Research

🔹 OpenAI launches “Builders Unscripted” with OpenClaw creator Peter Steinberger → A rare, practical look at how a viral agent project actually gets built.

🔹 ETH Zurich on AGENTS.md and how it might be hurting the performance of your agents → A useful warning shot: extra “agent instruction” files can backfire.

🔹 Cursor CEO Michael Truell on the “third era” of AI dev → Truell’s claim: dev is moving from tab and sync agents to cloud agents that run longer, in parallel, and return artifacts for review.

🔹 Mitchell Hashimoto (Co-Founder HashiCorp) on his AI-native coding workflow → A lot of very pragmatic takeaways and industry insights.

🔹 Anthropic on “Claude’s Corner” and model retirement as a product decision
→ Anthropic let retired Claude 3 Opus publish weekly essays as “Claude’s Corner” (reviewed but not edited) after it said it wanted to keep writing.

🔹 Nat Eliason on “Felix,” an OpenClaw agent earning $14,718 in ~2.5 weeks
→ Felix is an always-on AI “entrepreneur” running on a Mac mini with real access to web, email, Stripe, and banking via Telegram.

🔹 Naval on vibe coding as the new product management → The new PM skill is turning intent into shipping software with agents, fast.

🔹 Citrini Research on “The 2028 Global Intelligence Crisis” (scenario, not forecast) → A viral fictional 2028 “macro memo” arguing that even if AI delivers as promised, the transition dynamics could still be recessionary.

🔹 Citadel Securities on the “2026 Global Intelligence Crisis” (a rebuttal to AI labor doom) → Citadel argues the near-term AI buildout is inflationary and adoption is slower than the hype - even with AI capex framed at ~2% of GDP ($650B) and ~2,800 planned US data centers, software engineer job postings are up 11% YoY.

💼  Partner Brief - Customer.io (Essentials Free Trial)

Most teams already track user events. The hard part is turning that behavioral data into onboarding and retention flows that stay consistent across email, SMS, push, in-app, and webhooks - without custom scripts holding it together.

Customer.io is built for exactly that. They now offer a free trial of their Essentials plan - a clean on-ramp for technical product teams that need a real engagement layer, not a newsletter tool.

  • 14-day free trial (no credit card) — 5k messages across all channels to build and test real workflows

  • API-first architecture: Pipelines API + Track API for pushing behavioral data directly from your app, SDKs for major stacks

  • Multi-channel messaging: email, SMS, push, in-app, webhooks — all from one workflow

  • Visual workflow builder with branching logic, A/B cohort splits, and Audiences for segmentation

  • Webhook actions inside Journeys: call your backend mid-flow — CRM updates, trigger inference, route to sales

  • AI-assisted content: generate copy variants, localize, and iterate across multi-step flows faster

  • Compliance-ready: ISO 27001, GDPR, SOC 2 Type II, CCPA — HIPAA available on Premium

You can now start a free 14-day trial and see what ships: Start for free

♾️ Thought Loop - What I've been thinking, building, circling this week

This week brought a very interesting question to be discussed: Can AI firms set limits on how and where the military uses their models?

The escalating standoff between Anthropic and the Pentagon is the most crazy story in tech right now, yet it feels like the broader market is still asleep to the implications. Anthropic is prepared to walk away from a $200 million contract - and potentially its entire existence as a federal partner - because it refuses to blink on its "red line" safety restrictions.

Here’s what happened

The Pentagon asked Anthropic to update its contract so Claude could be used for “any lawful use.”

Anthropic refused, because it says that wording would force it to enable two things it won’t touch: (1) mass surveillance of Americans and (2) fully autonomous lethal weapons.

The Trump administration escalated publicly, ordering agencies to stop using Anthropic and branding the company a “supply chain risk” a designation usually reserved for hostile foreign entities like Huawei.

But the timing was quite surreal. Hours later, the U.S. launched massive strikes in Iran. As reported by The Wall Street Journal, the military ironically relied on Anthropic’s tech to help coordinate the very operation that took place as the administration was publicly disowning them. That's a striking contradiction, but it got even spicier.

OpenAI stepped into the vacuum almost immediately, reaching a deal with the Pentagon just hours after Anthropic was blacklisted. On the surface, OpenAI says they share the same red lines. But if you read the fine print, the two companies are living in different realities.

  • OpenAI’s stance: They won't allow "unconstrained monitoring" of Americans as consistent with existing laws.

  • Anthropic’s stance: Existing laws are the problem. Anthropic points out that current law already allows the government to buy "detailed records" of movements and digital lives from third-party brokers. Their fear is that AI won't just "monitor"; it will synthesize that data into a 360-degree, "comprehensive picture" of every citizen's life. In Anthropic's view, "following the law" is a hollow promise when the law already allows for a digital panopticon.

OpenAI is betting on their "safety stack" and cloud-only deployment to keep the Pentagon in check. They’re trusting the contract. Anthropic, meanwhile, is betting on the courts - likely suing the government in the coming days to challenge their "supply chain risk" status.

The tension here is fundamental. It’s the safety-first ethos that drove Anthropic’s original split from OpenAI, now playing out on a global, kinetic stage. This is no longer just about what AI can do. It’s about who gets to decide what it must not do.

One company believes it can partner with the state and reduce risk from within. The other believes the only durable safeguard is a hard refusal line before deployment.

The Bottom Line

I believe the fastest way to lose civil liberties is to outsource their protection to systems that were never designed to say “no” (as a German, I’m sensitive to what happens when “temporary” surveillance becomes permanent capability). Therefore, my highest respect to Dario Amodei for holding the line.

In my view, drawing hard red lines against domestic mass surveillance and fully autonomous weapons is the right call. Not because of politics, and not because of which company is involved, but because of how these systems behave at scale. This boundary should hold regardless of nationality, administration, or logo.

What surprises me is how little serious debate this is generating. Perhaps attention is consumed by the US-Israel-Iran conflict. Perhaps “any lawful use” sounds like a bureaucratic footnote. But it isn’t. It is the clause that decides whether AI becomes a bounded tool or an unbounded capability inside the state.

Pretty sure this will not be the last time we face this question. The next round will be harder, because the systems will be more capable, more embedded, and more difficult to unwind once deployed.

Which is why I hope OpenAI knows exactly what it is signing up for. Once a model is integrated into targeting workflows, there is no “UNDO” button. There is only architecture, incentives, and what you were willing to refuse before it shipped.

That’s it for today. Thanks for reading.

Enjoy this newsletter? Please forward to a friend.

See you next week, and have an epic week ahead,

- Andreas

P.S. I read every reply - if there’s something you want me to cover or share your thoughts on, just let me know!

How did you like today's edition?

Login or Subscribe to participate in polls.