Descriptions:
Video Description
An AI researcher left Anthropic warning that the labs are gambling with our lives, and this video from the Conscious Radical channel places that resignation beside an older story. Leo Szilárd pushed for the atomic bomb, then wrote a petition against using it that was classified before it reached the President.
The parallel is careful rather than hysterical. Builders of the most powerful technology in a generation are ringing the same alarm they once ignored. The video asks what happens when the people who built the machine are the ones most afraid of it, and why their warnings still struggle to reach the room in time.
Full Video Transcript
In 1945, Szilard wrote a petition asking his government not to use the weapon he helped build.
Yesterday, 27-year-old Jacob Coxon wrote a similar petition on X about AI and he quit his job to do
it. That notification came up on my phone this morning. It was an AI summary telling me about
an AI researcher who resigned whilst warning about AI. Now Jacob Coxon has spent the last
three years doing pre-training research first at open ai and then at anthropic now pre-training is
where the model learns and coxswain was one of the ones helping it learn but then he resigned
on the 8th of september and said publicly that neither open ai nor anthropic is acting
responsibly that they’re racing towards self-improving super intelligence and gambling
with our lives he told the wall street journal that in aggressive scenarios things could already
be out of control by the end of next year now some believe he is wrong about the timelines but still
the chief exec of anthropic put on record that the odds of this going catastrophically wrong are
between 10 and 25 percent the chief exec at openai signed a statement in 2023 saying that extinction
from ai should be treated like pandemics and nuclear war so we have an absolute right to
be concerned and yet despite these concerns they’ve gone back into the lab to build them
faster the reason they want to build it faster is that if they the careful people stop then the
careless people will continue anyway and isn’t it better that the careful people get there first
so they understand the danger perfectly but they keep going because stopping would let
a careless company get their first so they’re saying trust us not them we’re
careful they’re not we’ve seen this unforgivable argument before because we
have run this race before my immediate thought was the moon landings two
superpowers racing to the finish line but the moon didn’t do anything once we
got there a closer match is the Manhattan project they said that the
Germans might be building one so we had to if we don’t they will brilliant
people who understood the risks but were compelled to work faster because the only other alternative
was stopping and allowing a more reckless group to get there first. Germany surrendered in May 1945
but the race continued. Szilard who had initially pushed for the bomb to exist wrote a petition in
1945 which was signed by around 70 scientists asking that the weapon not be used on Japan
without a warning and a chance to surrender but that petition never reached
the president and then three weeks later there is Hiroshima we’ve also seen this
in films something gets out of the lab despite the scientists insisting that
it’s contained and there’s us on our sofa watching knowing exactly what’s
going to happen and where this is going remember the scientists in the film are
always ridiculously clever genius level they have funding and they have a
schedule and they believe so strongly in their ability to handle it and that the risk is
manageable mary shelley gave us frankenstein and his monster just one man a tower and a lightning
storm but in ai there are several hundred people each doing a sensible job each pointing at someone
else when it goes wrong the engineer points to the model the executive points to the engineer
the model points of the data now i am old enough to remember the 1983 film war games
i remember sitting on the edge of my seat panicking about the end of the world the
computer plays out every nuclear scenario and finally concludes that the only winning move
is not to play okay so surely there are guard rails well yes there are but the labs mostly
wrote them and they’re voluntary so i’m not sure i trust a player who writes their own rules
now the eu has brought in rules these include requirements for transparency safety assessments
oversight basically aiming to increase accountability for how ai is developed and
deployed and the uk has set up an institute that can run evaluations and tests on advanced ai models
with the idea to identify risk or blind spots but it lacks any legal powers to force the companies
to stop or change their work meanwhile america where the labs live has nothing binding at all
according to reporting one of these companies is preparing to list on the stock market at a
valuation of around two trillion dollars which means that your pension fund could be about to
become a part owner but what i see is a pattern playing out again and again having an enemy
permits us to take action and being quicker than them is always a virtue we’re told we can trust
them because they’re not the reckless ones so what can you and i actually do well i’m still working
this out but for starters we need better more independent guard rails now not next year we need
to ensure that liability for all it’s worth if something goes catastrophically wrong lands on
the company and the individuals heading it up we cannot have the excuse of the model did it
we need a clear agreement in place where race no longer comes into the equation and i appreciate
that that’s probably the hardest to achieve but i also know there is a trap inside their argument
fear is a product in fact fear is usually the main product we are much
easier to control when we’re frightened and people on every side benefit when we
are too scared to think but I will also say that this technology can be helpful
it can read scans it folds proteins that took decades to map and it gives a voice
to people who’d lost theirs and that is the problem it can be genuinely helpful
and that is where we need balance. I talk about ancient texts and I link them to the current
world and the main point I keep seeing in these texts is our need to pay attention, not just to
wake up or check out but to concentrate our awareness, to listen, to watch closely and question everything.
Source: The Conscious Radical