Met for Evals discussion meeting with Gabe & Xiaofan
Met separately afterwards where she essentially told me to silently fail
Meeting up for dinner at Vics in Noho for 6:30pm, originally res for 6pm
Said if I wanted to meet her CEO Drew and Isaac, originally said yes but then said no when asked for dinner because I was feeling over stimulated and not fully present
2025-10-11
Messaged regarding best way to approach evals, and said 3-5 tasks and then some perturbations on input
Discussing topic on evals, mentioned Anthropic’s blog says more data via synthetic data trumps data quality which is opposite of what I heard in fine-tuning
Heather thinks this is because down-sampling reduces more of the “cruft” and believes data quality is analogous to “no cruft”
2025-10-10
Messaged me early AM saying she’s got indication that teams will be re-worked and to ignore everything Xiaofan is saying.
I did though continue to work on finishing the Notebook reading prompt and mentioned I might work on MFR (Model Factor Residualization) because it was a direct ask by Sumi.
I then later heard from Heather that apparently DP agreed to split the teams into:
Team A:
Xiaofan
Ava
Yucen
Tim
Team B:
Heather
Gabe
Adam
(Potentially Sasha himself)
Team A will work on alpha delivery and team B will work on evals for the first two workflows, though Heather suspects DP wants Team A to only work in an eval driven approach.
2025-10-09
Setup quick meet to sync up after meeting with DP
Heather was back home and her kids were also on the call, Maxime & Feline
Tried to communicate issues with Sasha, where currently it feels like he has a lot of questions, architecture proposals, as well as opinions, but he does not listen, speaks over people and has not demonstrated NLP expertise to me or others
Heather is generally biased towards assuming high degrees of competence and being okay with people failing. She told a story regarding someone who used to work with her who was very smart and was pursuing a PhD in Systems. He had left his PhD with his final paper in review (? forgetting the stage but essentially it was mostly done and just needed to get it over the line). He went off and got super interested in the world of compilers, types and started putting out research related to that (I’m unclear of the details and even if this sentence is correct but essentially he went towards more theoretical research which he provided with axioms and formulas). Once back he similarly tried to put out research like this, but wouldn’t finish the last steps of making them runnable experiments. After a few failed papers, his advising committee (? whoever was accepting/rejecting his papers) told him to definitively to no longer put out new work but instead finish the current papers he has. Heather apparently had already given him this advice but according to her she felt like she was brushed off as “Systems people”
I believe Heather’s point of the story was to show the different work styles, and the difference between practical/theoretical. Where she may be alluding I was more on theoretical comparative to Sasha’s practical.
I thanked her for her story but said I think this doesn’t apply, because I don’t have an academic background and I’m actually very practical focused myself. I just personally want to follow someone I believe has done sufficient research into the problem statement, and in particular don’t want to follow someone’s architecture diagram when I don’t believe they’ve done sufficient POC to demonstrate the why of their design.
Heather understood and said she realizes that maybe Sasha has used up his political tokens within this group, but then had to leave because of kids.
I then messaged regarding where we were and essentially said I don’t believe I’ve lost favor with DP yet, and I did share some of my frustrations with the direction this project is going currently to Sasha. I believe Sasha will try to split the teams, which I agree with, but I’m not sure on Sasha’s delivery and wanted to know her opinion.
2025-10-05
Saw that she messaged me on Slack in the group between her and her PhD students
Spoke about DSPy being mentioned in Simon W’s blog
Heather mentioned she signed me up to experiment with Hector on CMU’s supercomputer and mentioned it wasn’t to pressure me into her project but more to get experience by working on a real system
Said I should read Hector’s paper that they recently released
2025-10-03
Met with her over teams in the afternoon, she was still at CMU
Meeting was regarding potential research directions to align with her work so I can transfer over to Labs
Started with priors regarding DP, Heather mentioned that in a meeting with DP & Claude, DP got very territorial when Claude mentioned what Andrei was working on
Said this was what she referred to when she said Claude said DP was more erratic and difficult to work with than ever
We spoke about possible research directions and I mentioned LLM FT and we came to the conclusion that this would be the best way to go because ultimately if a modeler is happy then the project is useful
Billy Li and Xianyuang Ding were introduced to me by Heather and we now have a meet set up this upcoming Tuesday
Heather told them I’m shy to ask questions and pointed me towards a research paper written by her student and Xuanyuang Ding as well as unslop framework that they used for FT
Having dinner on October 15th to discuss more about her startup
2025-08-28
Messaged Heather about Jupyter AI being done on my end and she was supportive and happy for me
2025-08-22
Said Jeff Wecker is still deciding what to ask the management committee in terms of requirements for LLM strategy. This blocks any next steps for potential LLM org
Asked her opinion of my message to DP and she said it was fine but if anything not strong enough
2025-08-19
Heather reached out saying the project doc by DP probably was nothing after finding out more
2025-08-15
Spoke in the morning about potential work reorg
Touched base on Sasha working on Claude Code Slack app and how I was surprised that’s where he was spending his energy. Heather mentioned that’s fine and he’s just getting his bearings.
Said she’d talk with Claude and see what’s happening