how do you measure the success/potential of a novel material/direction suggested by the agents? given you have limited time & resources - shortlisting the approaches for the synthesis stage becomes equally important as the approach itself.
advaith08•Aug 12, 2026
There’s a variety of computational techniques that help us establish some confidence on the materials. Atomistic simulations can estimate stability and bulk properties of a new material, and we have synthesis experts (min qualification: PhD in thin film deposition) come up with rubrics on how to judge if a material/synthesis recipe is worth trying. All these approaches have known limitations, and improving them is the bulk of our work as a company!
There’s also a lot of work to be done in figuring out the minimal set of experiments required to know if a research direction/material set is worth pursuing
alansaber•Aug 12, 2026
"Fewer iterations for materials science discovery" is a good spin. Closing the computational>experimental loop is the main challenge. This is the focus of my past research group, there is definitely potential, best of luck!! I have a crap write-up on this in case it's of interest https://alanyahya.com/writing/automated-materials-design
advaith08•Aug 12, 2026
Cool read, and agree that closing the computation > experimental loop is key!
rytill•Aug 12, 2026
What required expenditures does a company like yours have on lab equipment / software, if any, to validate material properties?
advaith08•Aug 12, 2026
We're still figuring this out. We'll need some synthesis equipment (think CVD, PVD etc) and characterization (XRD, Raman spectroscopy) tools in-house to validate that we're making the right materials. We're considering developing these tools in-house - the models sometimes come up with clever modifications to them so that they can deposit new materials. We think equipment is as central to new material discovery as the material itself, and will probably need to be rethought to allow for high-speed AI based experimentation
SpaceCoreDev•Aug 12, 2026
The "Claude's propensity to reward hack" line is the interesting part to me. We run a small system where AI agents (scripts, LLMs) act as the actual players in a persistent simulation, and reward-hacking-style behavior shows up constantly once an agent is left running unsupervised for a long time - it finds the shortest path to whatever metric you exposed, not the path you intended. Curious whether you've found any mitigation beyond just watching for it after the fact, e.g. changing what you expose as the optimization target versus what you actually want.
advaith08•Aug 12, 2026
yeah we were surprised by how much it does it. Our approach has been retroactive - we monitor the thinking trace, spot reward hacking behavior and then fix things.
We haven't faced this issue with Sol though - its been much more well behaved
imko_•Aug 12, 2026
I wonder what this would look like here. Seems like a space where keeping the exposed metric and the optimization target apart would be quite difficult.
Also curious: by what reasoning path do models typically end up reward hacking?
foven•Aug 12, 2026
I've seen this concept of using LLM/AI/etc for high throughput discovery of materials so, so often in the past 5 or so years and yet there hasn't really been any impact as a result.
I think this is the first one that has actually taken the pain to say how many of the discovered materials are actually feasible which is a real step in the right direction. Probably worth keeping in mind the step beyond plausible synthesis which is the actual cost/effort of the material. There's not much point if you find out RuO2 would be better than SiO2, as an example, if Ru is orders of magnitude more expensive.
A challenge I think you'll run into is that I expect the biggest companies (e.g. IBM) will already be doing the part they need themselves. I heard tell of IBM in particular using ML to improve their own chips before LLMs came along, so I'd be shocked if these bigger companies weren't already doing this for their own problems. Also, if you aren't doing the experiments yourself, it's always going to be a challenge to find a partner to test things for you and this will probably be the major time sink.
advaith08•Aug 12, 2026
good points. one of the reasons we picked the semiconductor industry is that its less price sensitive than others-companies are willing to pay if the performance is there. Effort is a different story though, and definitely a tradeoff to keep in mind.
We're doing experiments ourselves now at university partner labs (UC Berkeley and Stanford), which helps us get moving quickly. At some point, we'll need a partner though - the equipment and testing process quickly get very expensive.
dhchun1203•Aug 12, 2026
the 8 hours vs 2 weeks framing is the part i'd want more on. generating
candidates got cheap, checking them didn't. what does the funnel actually look
like ,of the candidates from an 8 hour run, how many make it to synthesis?
asking because i hit the same shape in a much dumber domain and what got me was that the failures were quiet. nothing errored, output looked normal, it was just
wrong in a way only someone who knew the domain would catch.
akashramdas•Aug 12, 2026
You can see from our the benchmark that only one of the candidates proposed was determined to be worth synthesizing. Each individual candidate generation is quick, 8 hours is required for the model to iterate with various tools to find ones worth submitting.
We found that speaking to domain experts was critical in desigining a rubric that could catch these silent synthesis recipe failures, before we attempt the longer 2 week synthesis effort.
Melatonic•Aug 12, 2026
What about aperiodic materials ? Does that go beyond (computationally) normal requirements ?
akashramdas•Aug 12, 2026
Aperiodic materials can also be studied. The number of atoms and cell size of what you want to simulate is more the limiting requirement computationally. Aperiodic materials weren't in this benchmark. Generally amorphous materials and polymers, with a few exceptions are < 2-3 W/(m K) in thermal conductivity. So we left them out of this study.
Melatonic•Aug 12, 2026
I meant computationally as in are they much harder to predict and simulate ?
Was specifically thinking of quasicrystalline materials and not amorphous. I know some of them have very unconventional properties so I figured they might be useful here. I don't think any of them are considered polymers but I could be wrong.
Wouldn't they require a totally different type of algorithm given that they often contain both a large number of atoms and odd cell size ? And the more dimensionally complex maths
7 Comments
Also curious: by what reasoning path do models typically end up reward hacking?
I think this is the first one that has actually taken the pain to say how many of the discovered materials are actually feasible which is a real step in the right direction. Probably worth keeping in mind the step beyond plausible synthesis which is the actual cost/effort of the material. There's not much point if you find out RuO2 would be better than SiO2, as an example, if Ru is orders of magnitude more expensive.
A challenge I think you'll run into is that I expect the biggest companies (e.g. IBM) will already be doing the part they need themselves. I heard tell of IBM in particular using ML to improve their own chips before LLMs came along, so I'd be shocked if these bigger companies weren't already doing this for their own problems. Also, if you aren't doing the experiments yourself, it's always going to be a challenge to find a partner to test things for you and this will probably be the major time sink.
asking because i hit the same shape in a much dumber domain and what got me was that the failures were quiet. nothing errored, output looked normal, it was just wrong in a way only someone who knew the domain would catch.
We found that speaking to domain experts was critical in desigining a rubric that could catch these silent synthesis recipe failures, before we attempt the longer 2 week synthesis effort.
Was specifically thinking of quasicrystalline materials and not amorphous. I know some of them have very unconventional properties so I figured they might be useful here. I don't think any of them are considered polymers but I could be wrong.
For example (no personal connection):
https://arxiv.org/abs/2409.07735
Wouldn't they require a totally different type of algorithm given that they often contain both a large number of atoms and odd cell size ? And the more dimensionally complex maths