I enjoyed the feeling when I made this connection talking with a startup doing this a while back. It&#x27;s just a different &quot;language&quot; and although it&#x27;s not a given that LLMs can operate in it, it&#x27;s a reasonable thing to try, and it turns out they can.

&gt; Learning a transformer language model on such data simply learns a distribution over sequences of tokens.If statistical distributions can model higher level polypeptide structure, then it could be useful.

yeah but your training is bottlenecked by the lack of ground truth. Some things were(I presume)&#x2F;will be easy to do with LLMs, like protein structure, because every part of every protein is source data (and there&#x27;s millions of known structure). But suppose you want to estimate clearance, or ld50. How many proteins do we know their serum clearance? 1000? 10000 maybe?

It&#x27;s not an English LLM, but a &quot;protein&quot; language model, where tokens represent amino acids or nucleotides. Learning a transformer language model on such data simply learns a distribution over sequences of tokens. It&#x27;s a fine approach conceptually that in many ways is the &quot;right&quot; way or most elegant method, and not a stretch at all.

I don&#x27;t have a direct answer to your question. My guess is that LLMs are too limited to make truly great solutions in biology but sequential modelling is a key component that will not be replaced any time soon. For example, transformers were key to AlphaFold&#x27;s success, but they still needed many other steps to make accurate predictions.I worked on a predecessor to LLMs - HMMs for protein modelling. They were, and still are for most people the best way to model protein sequences. It&#x27;s usually done as prediction, rather than generation (IE, you use the model to classify an unknown sequence into a known category, rather than asking the model to generate new instances of a category). HMMs for proteins are a bit stuffy, and they model local changes well, but struggle with long-range interactions that LLMs seem to excel at (for example, an HMM will do a good job of letting you stuff a few more residues into a protein in a localized region such as a hinge, but are not so great at modelling groups of residues that are located far-apart in sequence space but close in protein space).One detail of the bitter lesson is, imho, that statistical parrots are better than they &quot;should&quot; be, probably for the same reason that mathematics is unexpectedly proficient in modelling physics: to some degree, the models recapitulate the true latent space of the underlying system well enough to generalize outside the original observation space.

I think your intuition is off here. The number of sequences to enumerate is much greater than the number of atoms in the universe. You need a smart way to enumerate these and that&#x27;s what the LLM is for. The statistical parrot is not a detour its a shortcut.

The real power of LLMs is they can model anything as a “language” given the right sequence training data.Warning: the following is my opinion.In the same way that MLP “neurons” are universal approximators, it seems that LLMs are universal mappers.They have the potential to help us organize and translate the immense quantity of data being generated by modern methods in all respective disciplines. We might create a model that translates english to protein synthesis, and vice versa, which would be pretty useful given my lay understanding of biochem.To your point - this probably is NOT the best way to do this in an objective sense. But to my mind we are hitting upper limits as finite beings and need things like this, which utilize native language constructs, to move forward.

First the search space is way too large for brute force enumeration. We’re talking like 10^300 combinations. Also the hard part isn’t just listing amino acid sequences, its finding ones that do what you want them to. The only way to figure that out is by testing them, which is difficult and expensive. So you need an algorithm that is good at only listing sequences that are likely to work. That’s precisely what LLM’s are good at: finding patterns and sequences that are correlated in a useful way

Well hopefully it&#x27;s trained on genetic DNA sequences and not Reddit threads. If so, it should do pretty well predicting the next sequence given previous sequences. There are probably all sorts of undiscovered patterns.

To be fair, having AI in the title landed it on the front page of HN, so...

Reading their blog post I wonder if an LLMs is really the best way to do this. If I got it right, they used the LLM to enumerate potential protein DNA sequences. Does that really need an LLM? Enumeration is not novel, nor are LLMs particularily good at it. If you want to computationally parallelize the search in a large enumeration space it would be much easier to simply, well, do that instead of taking a detour via a statistical parrot.In a nutshell this sounds more like a case of &quot;we wanted something with AI in the title&quot;.

Is this going to be as good as when AI arrived in the world of materials science?<a href="https:&#x2F;&#x2F;www.404media.co&#x2F;google-says-it-discovered-millions-of-new-materials-with-ai-human-researchers&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.404media.co&#x2F;google-says-it-discovered-millions-o...</a>Or is it only just going to be as good at generating headlines?

And their repo: <a href="https:&#x2F;&#x2F;github.com&#x2F;Profluent-AI&#x2F;OpenCRISPR">https:&#x2F;&#x2F;github.com&#x2F;Profluent-AI&#x2F;OpenCRISPR</a>

Gift article (no paywall): <a href="https:&#x2F;&#x2F;www.nytimes.com&#x2F;2024&#x2F;04&#x2F;22&#x2F;technology&#x2F;generative-ai-gene-editing-crispr.html?unlocked_article_code=1.mk0.T7-S.0EFqiNdALV08&amp;smid=url-share" rel="nofollow">https:&#x2F;&#x2F;www.nytimes.com&#x2F;2024&#x2F;04&#x2F;22&#x2F;technology&#x2F;generative-ai-...</a>Preprint: <a href="https:&#x2F;&#x2F;www.biorxiv.org&#x2F;content&#x2F;10.1101&#x2F;2024.04.22.590591v1" rel="nofollow">https:&#x2F;&#x2F;www.biorxiv.org&#x2F;content&#x2F;10.1101&#x2F;2024.04.22.590591v1</a>

We are still early. Eventually you&#x27;ll be able to change your race, gender, add reptile eyes, regrow limbs etc. Has to start somewhere. Need more data.

You mean exactly what 23andMe tried to do, and failed miserably at.

I like this a lot; you could have a multimodal setup with a DNA transformer, an image transformer and an LLM. Extremely fundable startup.

Imagine an AI learning from photos&#x2F;videos of a person and their DNA sequence? And also a list of diseases, health records, etc. Then asking it for predictions while giving it feedback afterwards so it can tune itself.You could even guarantee privacy. That would be some really useful data.

The real endgame here isn&#x27;t to just enumerate and then patent those sequences, right?

Captain TripsOne of the four horsemen of the AI apocalypse.

So the six finger hands were just a foreshadowing?

More and more I find the flippant dismissal of the amazing breakthroughs that are constantly streaming in kind of disheartening.Nobody is impressed by anything anymore, nobody is excited about the future - everything’s just “terrible” and everything looking forward is bad. I hate that mindset and I’m sick and tired of cynicism poisoning practically every online space. Thank you for saying it’s “lowbrow” and giving OP a little bit of a hard time.

This model has nothing to do with ChatGPT other than transformers. And as someone that could desperately use some advances in gene editing, this lowbrow dismissal is frustrating.

Have you seen how generative AI thinks hands work? Just a few edits and reality can catch up.

It&#x27;s one thing to analyze it. It&#x27;s an entirely different thing to let a machine of dubious abilities create new DNA.

Have you seen DNA? Mistakes and dupes and hallucinations all over the place. Ever since Sherlock Crick and Doctor Watson started meddling with it.

What can possibly go wrong if we let ChatGPT edit our DNA?

Computer viruses run on hardware they do not create but merely hijack. Unless a virus took control of a semiconductor fab, it&#x27;s hard to argue that they are alive&#x2F;reproducing in the context of this discussion.

I think of robots as being physically embodied somehow. I don&#x27;t think of a software program like a virus as being a robot.

I still consider biological life as the best ‘robot’ because it can create more of itself.As long as robots are incapable of recreation I don’t see the threat.One could say all maschines today are infertile.

Do you mean for AI to do the entire job of researching, creating, testing, manufacturing, and distributing a cure, or just for AI to be involved? And do you mean completely eradicating a disease, or just producing a cure for it? And do you mean an outright cure, or also a treatment or vaccine? If the latter in all cases, here&#x27;s an example:<a href="https:&#x2F;&#x2F;www.technologyreview.com&#x2F;2022&#x2F;08&#x2F;26&#x2F;1058743&#x2F;i-was-there-when-ai-helped-create-a-vaccine-covid-moderna-mrna&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.technologyreview.com&#x2F;2022&#x2F;08&#x2F;26&#x2F;1058743&#x2F;i-was-th...</a>If you mean the former in any case, it&#x27;ll probably be a while, if ever in our lifetimes.

It cured my investment interest in it.

Die Instantly Man is the worst superhero...

Generative A.I. arrives in the gene editing world of CRISPR