An Interview on the Journey from U.S. PhD Applications to AI Industry Jobs | Original
Source: chatgpt.com
A: Zhiwei Li, Software Engineer working at the intersection of AI and finance.
B: Haifeng Jin, Software Engineer working on AI Infra.
The following conversations have been corrected for grammar and lightly refined by AI.
Table of Contents
- PhD Motivation and Application Process
- Primary goal was securing a U.S. job
- PhD funded via teaching or research assistantships
- Cheaper than master’s degrees with financial aid
- High volume of professor outreach required for admission
- Navigating PhD Studies and Advisor Selection
- Switching advisors cost an extra year
- Balance professor productivity with their supportiveness
- Utilize insider info from current students when choosing
- Research, Coding, and Career Goals
- Coding skills crucial for research and industry jobs
- Publications not always required for PhD graduation
- Open-source projects can demonstrate expertise effectively
- U.S. Job Market and Global Opportunities
- U.S. offers access to top tech projects
- Internal company transfers easier than new grad applications
- Current entry-level job market is highly competitive
- TOEFL, GRE, and Language Preparation
- TOEFL and GRE each taken three times
- GRE vocabulary significantly harder than TOEFL
- Test scores sufficient for many university applications
- Open Source Development and AutoKeras
- Prioritize usability over innovation in open source
- Early project launch increases success likelihood
- Major refactoring sometimes necessary for project clarity
PhD Motivation and Application Process
A: I’m in the car with my two phones using a Macau SIM card. Can you hear me clearly?
B: Yes, I can hear you clearly now, and the video is smooth.
A: Great. Let’s turn off the camera if the network isn’t good. Also, could you increase your volume? My side is at maximum. I’m recording with my second phone for transcription.
B: I’ve increased the volume. I’m fine with turning off the camera if needed.
A: Perfect, thanks! By the way, you look as young as when we met years ago.
B: Thanks, that was about six years ago, 2019.
A: Cool. Can you open my website, lzwjava.github.io, and share your screen?
B: Sure, let me send it to my computer. I need permission to share my screen; it says the host disabled attendee screen sharing.
A: Let me adjust the Zoom settings to allow all attendees to share. Try again.
B: It works now. I’m sharing. Is it clear, or should I zoom in?
A: Yes, I see it very clearly.
A: So, I asked that what… I think for this question, because we are friends, it’s not so formal. If you find something very easy, or you repeat too much, or if it’s something that can just be searched on Google or ChatGPT very easily, we can skip it. I haven’t yet compared your knowledge about it, and I might be missing some information. Also, from my point of view, from an AI point of view, the context of prompt engineering might not be good for this. So we may have whatever you want to say. If you want to share more, then you can share more. Okay?
B: Sure, sure.
A: Cool. Well, first, what motivated you to pursue the PhD in the U.S.?
B: I’ve been asked this a lot. The number one motivation was to find a job in the U.S. job market, to work for one of the best tech companies in Silicon Valley. That was my number one goal. A master’s degree can also do that, but a PhD is usually cheaper with assistantships or scholarships—less financial burden on my family. So that’s why I decided to pursue a PhD instead of a master’s. My goal is similar to many other students who pursued a master’s in the U.S.: to find a job in the U.S. with good pay.
A: Cool. I want to extend: how many years was your PhD and what was the total cost?
B: It took me six years to complete my PhD. Five years is standard. I took six because I switched an advisor once after around one year, so I needed to start fresh for another five years. So, totally six years.
A: First, the tuition cost for five or six years. Second, the living cost.
B: If you get an assistantship, like working as a teaching assistant or research assistant, your tuition is waived entirely. You don’t need to pay anything, and they pay you monthly. I worked in a rural area, one hour away from Houston in College Station for Texas A&M. I got paid $2000 per month with no tuition. It was enough to cover my living cost; I could even save some money. But many people go for summer internships and earn more. The main expense came before going to the program: application fees, air ticket, first month’s rent, and deposit. These need to be paid beforehand.
A: That cost, like 100k CNY? Would 100k CNY cover that cost?
B: Let me think. 100k? Yes. I didn’t use a good agency; I prepared my application and mailed data on my own, which saved a lot of money. I spent around 40k CNY.
A: 40k. Okay, cool. You paid like an extra 50k CNY for an agency to help prepare the application, then it’s almost 100k, right?
B: Ever. I have a request. Could you open your iPhone and use the memo app to record? My phone is recording, but the voice may not be very good from my side. Could you record on your side and then send the voice memo as an m4a file after we finish? You are using your mobile phone; could you open it and also record on your side?
B: Sure, let me check how to record on my iPhone.
A: It’s in Voice Memos. Just click the red button.
B: Got it. I’m recording now.
A: Thanks! I appreciate it. Sometimes I accidentally stop my recording, and losing it would be a disaster for the interview.
B: No worries, I’ve got my side covered, though it’s just my voice.
A: That’s fine, my questions aren’t the important part.
A: Okay, I got your tuition and cost info. Agency and DIY—you are self, okay. They say the… I want to ask: you also told the GRE. You are okay. The second question is: you used to be at Beijing Forestry University, BUPT, ACM ICPC. Share your decision. You got a silver medal in China region? The context: you got a bronze medal in ICPC. I think going to BUPT, that’s like. were you a medal winner or whatever? The star was out before you went to Beijing Forestry University? Did you want to go abroad? Or were your good achievements in university what made that decision? Or were you lucky before going to the U.S.? I think you saved many troubles. So I think that’s smart. How did the journey before going to the PhD impact you?
B: First, I didn’t know anything about studying abroad before going to Beijing Forestry University. After I went, I saw a lot of classmates preparing for GRE, planning for master’s degrees abroad. I found it interesting but didn’t decide to do it myself until I got a bronze medal in ICPC. I needed to find my next career goal. I already got what I planned when I first entered college—I decided to get that medal, and I got it. After some research, I found studying abroad could be a very good thing, mainly as my next goal to find a good software engineering job. That’s why I decided to study abroad as a method to find a good job.
A: Cool. You got the bronze medal in the third year?
B: The third year.
A: Number three here. I remember you went to Beijing Communication University for a master’s program, and then you came back to Beijing Forestry, room 302, and we met the first time there.
B: Okay, then you got… I definitely didn’t have the bronze medal at the time you remember.
A: Do you remember that day? I remember you came back to BGFU to handle something, to visit the 302 seminar room. You came back that one time. You had just graduated that summer, and I went to Beijing Forestry that summer too. That temple, oh, nice.
B: That’s amazing, you remember so clearly.
A: I just record it. You not only one time; probably you handled your GPA or whatever. You sometimes went back again, right? Sometimes two or three times. After you went to the master’s program, you still went to the bachelor university sometimes to handle GPA or whatever? No, my GPA. All the recommendation? I don’t know.
B: Sorry, what’s the question? After you go to the national program in Beijing Communication, you still sometimes go back to Beijing Forestry University, based on my memory.
B: Yeah, yes, I did.
A: Why did you go back?
B: For fun, you know? There were programming competitions and getting new members into the club. Mainly for fun—nothing related to my PhD application. It was just for fun.
A: You did the master’s program for two years or three years?
B: The master’s program should be three years, but I went to Texas A&M at the end of my second year. My master’s and PhD programs overlapped for one year. I finished all my courses and got all the credits needed in two years, so for the third year I didn’t have to be on the BUPT campus.
A: So you went to the U.S. around 2016 or 2017?
B: It was 2015.
A: 2015, so early. You graduated around 2020 or 2021?
B: Yes, 2021.
A: Okay, got you. I have been to the U.S. two times. One was early, like March, the season. I basically went to Silicon Valley. The second time, in 2017, I went to several countries like Silicon Valley, Seattle, New York, and Niagara Falls. I didn’t get to meet you because I tried, though probably.
B: Yeah, I remembered you were in Las Vegas when the mass shooting happened, right?
A: Yeah, exactly. I just got off the airplane, and the TV at the airport… I went to a hotel nearby. At that time the shooter wasn’t found, but I was so curious I went to the area around the mass shooting, like 100-200 meters. The police asked me to go back, to fuck off, because they were busy with the shooter. People were crying. There was a Chinese basketball team player there. The tables were closed, people crying, calling phones. People were shooting all in the hotel. It was shocking. You didn’t know where the shooter was; they might be just in your next room. There were a lot of cops. That’s crazy. At night I had to keep the lights on in my hotel. I finally got to a hotel and kept the lights on. The shooters? No, no fun. I found it very hard to sleep and was very scared. At that time you were in Texas A&M University.
B: Right, right.
Navigating PhD Studies and Advisor Selection
A: Could you publicly share? I think you can also avoid this question if the switch of your mentor is not a very good thing. Could you share the reason why you switched? I think if you want to go back to your decision, you also… because switching cost you one or two years. If you had no other choice, though, if you select a mentor more carefully, you won’t be switching. So I think the reason for your switch is very important for people who go to the U.S., right?
B: Yes, I think it is important; definitely something to learn. The main reason was that my standard for picking the right professor was not so smart at the time. During my study at BUPT, I saw a lot of professors push their students really hard, and students had a hard time doing research and graduating. I never wanted to find a professor like that, so my standard was to find a nice person—that was the number one criteria. I talked to a handful of professors and picked the nicest one, who was my first advisor.
The problem was that professor didn’t want to maintain her lab anymore after one year or so. I’m not sure why. What she said to me was that she wanted me to pursue what I was really interested in, but I think the real reason was she found it too much work to maintain a research lab. She was super nice but also not… when a person is super nice and never pushes their students, they may not work hard either. She found it too much work and asked everyone to switch advisors. So I switched to another advisor. One thing I learned: you need to find a balance. Not just look for the nicest professor; look whether they are productive or not. The ideal goal is to find someone nice to their students but also hardworking and productive. If you can’t, try to strike a balance between these two, not optimize for just one.
A: Okay, I think also this is hard to know beforehand, right?
B: Right. You can only try to tell from their record of publications and through the interview. If you know someone in that professor’s lab, you can ask directly.
A: This is about several things: general observation of how nice, how pushy, the productivity. So that professor was very nice, talented, but maybe had an easy life, so the work didn’t occur, she felt overwhelmed, and made the decision to close the lab, right?
B: Right, that’s my understanding. She didn’t say it herself; that’s my understanding.
A: This adjustment happened in the middle after you joined Texas A&M for two years, right?
B: For one year. So this is…
A: These adjustments happen early.
B: Yes.
A: What’s the difference? After you joined for one year and found a new advisor, was it easier to find one? Or was it still hard to find?
B: The second one was much easier when you are in the university. You have much more information to find the right advisor—the personality of all potential advisors. You can pick the right one, and you may also know their requirements for choosing students. So it’s likely you can pick the right advisor and know how to pass their interviews to join their lab.
A: Cool. I want to review our core and our pace. How long today? In your company, or is it middle non-rest? Are you okay to talk for one hour or half an hour or 10 minutes? I…
B: I have another maybe half an hour or so fully free. Today is Labor Day, a holiday.
A: Okay, so it’s a holiday. You still have another half hour. Okay, let’s be quicker a bit. The same routine: I have the note to do. There’s a recommendation later, maybe two or three, and then GPA, and then TOEFL score, and then GPA, and then recognition. And then you approach the advisor professor, and then if you apply and get the email, you accept, and then you can use that for the immigrant or F visa, and do all that thing, and then you can go, right? What’s the major? The meeting may be ended due to some… I will set up a new one. Okay, sure. If they end us, I will settle new. Now we still can talk.
B: I can briefly go through how I prepared the application, how I got this. I didn’t use any agency. I mainly wrote everything myself and used some English revision or writing help service on Taobao. They helped me revise letters. I also bought some service widely used… I believe they have some popular survey they use. Basically, I wrote everything myself and had someone help me revise, which is cheaper compared to a full agency service. For the visa, I just filled in all the forms and mailed out the documents. Those are simple. If you want to save money, there are ways, but I don’t think you need to use those; it’s not a big deal. The main cost is the application fee, which you cannot avoid.
A: Application fee for one university is like 100 or 200, right? Around 100 or 200.
B: Yes, I believe it’s around 50 to 150.
A: 50 to 150, okay.
B: I filled in all the forms and mailed the documents but still didn’t hear back from many universities. So I started contacting professors. The reply rate was pretty low. For a PhD application, it’s something you have to do. When you finish the paperwork and mail it out, you need to contact professors. The reply rate for me was: when I sent out 10 emails, I got one reply. Among that, the acceptance rate was 10%. So for all emails I sent out, it was a 1% acceptance rate—that professor showed interest in working with me. You have to send a lot of emails unless you have connections. It was hard for me mainly because I didn’t choose the right list of universities to apply to. I only applied to very good universities, so I had a hard time contacting professors. If you correctly estimate what universities you can get an offer from, you don’t have to send that many emails.
A: You mean you should choose universities like Texas A&M level. You also chose a lot like Stanford, MIT. You overvalued your candidate; you thought you worked hard and were able to go to the top, but from the professors’ point of view, you were mediocre, normal. They see many good talents, so they have high selection.
B: I overestimated myself because the people I asked advice from were like you—when I tried to apply for a PhD, I asked a lot of people I know who did great in their career or application. They were all super talented. When I showed them my list, they didn’t say I overestimated myself because when they prepared their list, they felt it was just right. They never felt they were not good enough, so they couldn’t assess whether the list matched me. When you seek advice, you always ask someone better than you, which introduced a bias toward better universities. That’s why I overestimated myself. Don’t take anyone’s advice 100%; look into your own situation and make the right decisions.
A: Okay, got you. So what I want to fast forward: the above PhD or picture first PhD application. If you did now, if you now… you think maybe my case is different: Beijing Forestry dropout, now getting an associate degree, like 10 years disappearance in China. But if you were now in the BUPT master program again, with your background, what would you correct? What three points or very short five points would you do better? How would you do it?
B: You may afford the BUPT program.
A: If you are now, you just go back 10 years ago.
B: How I could better prepare for my PhD?
A: Yeah, how would you choose your routes? We say several things: one, choose advisor wisely, better. Second, choose mid-level or Texas A&M or below level, a wider range, not just the top. That’s the unnecessary effort, right? What other points would you add?
B: Don’t have much. The best case is to have direct connections with any students already in a PhD program. Get information from them about any professor; that would be super helpful. I applied blind without any guidance or insider information, so it was super hard. I had to apply to a lot of universities. If you know which lab is recruiting and the situation of the lab, you don’t have to apply to many; you can focus on a few with the highest chance. That would help direct your effort. It depends on your situation; I didn’t know anyone in a PhD program then, but now information is easier to obtain. Anyone applying now can try to get more information.
A: Okay, got you.
A: Hi, hi, hi.
B: Yeah, that’s a continued discussion.
Research, Coding, and Career Goals
A: Part two: since you enjoyed the coding, audio about this guy. What were the moments you thought of the critical switch pattern? You get three. Since you enjoy coding more than writing papers, how did you balance that? I think you once mentioned in your public information that you don’t like papers, but later you did AutoKeras. When doing AutoKeras for some years, you actually made a very good paper about it. So you finally did a very good paper, but you used… so the story is like that. How do you see paper and coding? How did your interest switch in your PhD journey?
B: Coding or software engineering is always important. It will help you in research, job seeking, or later career. For writing papers, you need to be working on the right topic and have the right way of doing research, influenced by your peers and advisor—the environment you’re in. I don’t think I did a very good job in research mainly because I didn’t have a passion for being a superstar in a research topic. I didn’t have such a dream like many other PhD applicants or students. My goal was always to find a well-paid job in the U.S. Papers would help, but only one aspect of the overall evaluation. I didn’t care about publishing super great papers; I cared more about how to get into big companies. I don’t think any PhD student needs to worry about paper publications too much. As long as you are a good student, can do basic courses, know how to code, understand logics, reasoning, how to write, you can meet graduation requirements. You don’t have to be super passionate or super good at writing research papers to graduate.
A: You also did open source projects; that can let you gradually write, not need a very high standard for papers, right?
B: For Texas A&M, I don’t think there were hard requirements for publications. Anyone whose advisor thinks they can graduate can graduate. Of course, they need to pass defense, but usually the committee agrees with the advisor. Basically, it’s the advisor’s decision; no hard requirement.
A: Your PhD thesis is about AutoKeras.
B: In general, but AutoKeras is definitely the most important part.
U.S. Job Market and Global Opportunities
A: Cool. My IELTS score is 6, okay? 2022, my IELTS score is 6. Now three years pass, I work, study, use English. This is my case. I dropped out of Beijing Forestry after one year education. Now the nine courses are passed from Guangdong Foreign Studies University, majored in computer science. I still need seven courses—four are computer science, like linear algebra or basic electronics. I still have seven courses not passed. I have around 10 years disappearance in China, working as an contracter engineer for DBS Bank, HSBC Bank, financial projects. I also have my startup and did some iOS/Android engineering. Do you think I must finish my associate degree? I have an associate degree, not bachelor’s. There are two ways for me. How would you suggest? I want to send the purpose with you to go not to the U.S., but to work in Hong Kong, Singapore, UK. I want to work overseas to enjoy the internet freedom. I want to bring my family outside. What ways do you suggest?
B: If you want to go my path—applying for a master’s or PhD program overseas and find a job after—a bachelor’s degree is required for any master’s or PhD program. You can double-check. This path is getting harder; too many people are applying, increasingly competitive. For top 50 or 100 universities in the U.S., a lot of students are from Chinese universities, filling up admissions.
The other way is to transfer in your job. For example, if you work for Amazon or Google in China, you can transfer to other countries if there is opportunity. This might be easier because you already prove you can create value. You are more exposed to opportunities inside your company. It’s more suitable because for master’s/PhD programs, they care about background so much; you may not have the advantage competing with others. Most importantly, when you graduate a master’s program, it’s super hard for a new grad to find a job right now in the U.S.
A: I don’t know that, because I’m not in the U.S. I heard of some hardness, but not… you say it’s so hard.
B: I didn’t apply for a new grad job for years, so I don’t really know the job market now. From what I heard, it’s super hard. The entry-level jobs are mid and senior. Those that use AI are very real, welcome. Latency, program engineer welcome. Also, starter, people now more wisely like AI. If you do AI research or AI engineer or very hardcore, people become… the remaining companies in the U.S. are quite selective because they fail; they started 10 years ago. So employers become wisely accept very high standard.
B: Yes, it’s also because of the economy. Every company is trying to reduce cost, including employee pay and number of people hiring.
A: I also observed that, like some banks, they increased some positions in China but cut down Singapore/Hong Kong positions because they are high paid. Recently, a lot of businesses closed shops because of COVID and the model of obesity doesn’t work now. Okay. Another: many times you would try TOEFL and GRE. Did you take IELTS? I also.
TOEFL, GRE, and Language Preparation
A: Did you take TOEFL and GRE? How many times did you take each, and what were your scores?
B: I only took TOEFL and GRE, each three times. For TOEFL, my first score was 92, and my best was 107. For GRE, my first score was 315, and my best was 326.
A: How much did each test cost, and were you disappointed with your initial scores?
B: TOEFL cost about 1500 CNY per attempt. I wasn’t too disappointed with 92 on TOEFL; it was good enough for many applications, but I wanted a higher score.
A: How much time passed between each attempt?
B: I took each test once per year over three years, starting from my third year in college through my master’s.
A: Also, you go to the U.S., you are able to listen to all English classes? In speaking and listening, daily use, how did you learn English actually?
B: It doesn’t help. College doesn’t help that much in daily life in the U.S. Somehow it helped me pass the teaching assistant exam—spoken English test before I could get a teaching assistant job. Otherwise, you can be a research assistant. For taking courses, understanding lectures, I don’t think TOEFL helps that much. It’s designed to help, but it’s too easy compared to actual university lectures.
A: Basically, you are… my IELTS is like 6, score around. Based on your understanding of my level now, communication with you, could I get 100 score?
B: Yes, I think 100 is not that hard to reach. You can definitely get that.
A: Thank you. How was the first year in your PhD? Very hard, right? English, new environment, a little harder?
B: Yes. I almost lost my teaching assistant job because of a mistake I made. I went to interview and asked someone to cover the class for me, but the way I did it was not appropriate. The professor thought I didn’t do a good job. I experienced trouble continuing my job, but finally everything turned out okay. One lesson: I had to be extra responsible for any job assigned. People care about that a lot; there will be consequences if I didn’t do it right.
A: Okay, got yeah. The difficulty: do you think GRE is saying the difficulty about double or half?
B: Different aspects. TOEFL is mainly evaluating English. GRE vocabulary is definitely double as hard as TOEFL. The math part is easier; a good Chinese student wouldn’t find it hard. But vocabulary, reasoning, writing are definitely double as hard as TOEFL.
A: Okay, so basically TOEFL, GRE is about English and math, right?
B: Yes. The English part is mainly reasoning, reading comprehensions, to measure your logic understanding.
A: About motivation: I found that you went through so much trouble for the PhD, and it’s okay. I want to say: I know someone in my surrounding, friend, hosted my startup Fun Live. He shared iOS knowledge. He graduated around 20 years ago, studied at that time.
He has a classmate who was graduated from Beijing Normal University too. The classmate went to the U.S., Seattle, worked at Microsoft Beijing Office for 10 or five, eight years, then transferred to Seattle. He got a disease, cancer or whatever, and died there. He just went to the U.S. for one or two years to die. There are some news about people going to the U.S. and getting depressed. But many people, most, have a good life.
My question is: now I know engineers, and I probably am good at the proxy. So information, I think you and I don’t have that much accessibility difference. My Apple Store U.S. account, open source, all apps I installed like 500 apps. Why did I take path like you? Do you think that your U.S. choice for the long run. If you live in 1970s or 1980s, I think your effort to go to the U.S. is super worth it because you have more money. You can do startup, do good things, you have more freedom. You live in the U.S., and there are a lot of cheap, China products.
I live in Guangzhou, work as an HSBC contractor, kind of high salary job based on the local standard. Why did I travel? What’s the most benefit if I spend next three years focused on this first thing, besides my job, all leisure time do this, Consider the best path. Why we all go to U.S.? This thing open source, GPT cloud is good. You go there, but also like in China, Tencent, many people under one million yuan a year. So, my question: do you ever think you never regret? No regret. I think you are living a good life, congratulations! You are happy to go to U.S., right? For normal young Chinese people, does it deserve to go to U.S.?
B: If your goal is to access the best technology, definitely recommended to go to the U.S. Unless you are in one of the best teams in China, like DeepSeek or working on Douban or best products in China, then definitely you don’t have to. But to my understanding, it’s super competitive to get into these teams. Even talking to them, I felt they hold a very high standard for hiring. Going to the U.S. is a good way to put you in a less competitive environment. You can get access to better projects.
In China, many good talents compete for a few good projects. The ratio of good projects over number of talents competing is much higher in the U.S. There are many good projects: Gemini, OpenAI, xAI. Good companies, but number of talents in Silicon Valley is not as many as in China. Just number of people competing for positions—the ratio is much higher in Silicon Valley, meaning you can get access to better projects. Another aspect: a lot of people move to the U.S. care about money. I don’t think it makes a big difference working in China vs. U.S. because services and goods in U.S. are more expensive. Even if you earn three times more, you can only buy similar services, foods, goods.
The actual difference lies in the hours you work. In the U.S., you can work fewer hours to afford same life quality. You definitely last number hours in exchange for that service. This is important for career because in the long run, anyone needs to pivot in their career. For example, I graduated specialized in automated machine learning, but now pivoting to machine learning systems, closer to hardware compilers. Software engineering is my core skill. This pivot will happen again somewhere in the future. We always need to pivot, and for that we need more free time to prepare. In the U.S., you have more spare time, which is crucial.
A: Pivot, how to spell it? Sorry.
B: P-i-v-o-t. Pivot transition.
A: Okay, I got. In the future, you want to work on ML systems. I also see some people say the U.S. is kind of easy life, and they go back to China like Wang Xing, Meituan founder, and TDEngine, time series database founder. He went back; he was at Motorola 20 years ago in U.S., went back to Beijing, started several startups, now does open source TD engine, maybe wealthy worth 10 million or more. He’s able to travel to U.S. or discuss business worldwide. So, what’s your plan for long time? Just come to China for holiday? Or consider later life go back to China for long stay.
B: Either way could be possible. The main deciding factor is the overall ecosystem in China. If there are more startups, companies doing good job in technology, then definitely possible to move back. Now there are a few good ones: Huawei, DJI, Taylor, also mothers like TikTok. A few good ones. If overall ecosystem gets as good as in U.S., I will consider moving back for career purposes.
A: You are doing ML systems, right? So do compiler, do ML the deep assistance, okay?
B: Yes.
A: For my background, like 10 years, three-four years in corporate, three years freelancer. I also enjoyed full stack like five-six years, and two years machine learning. I did GPU, got two certificates from Coursera. In my background, why do you think ML system is the best for you? I also find what’s my next recently. I am very excited about using cloud holder a lot to do coding and learning, like one month, one million codes, several computer app. All open several cloud holder instances or Codex from OpenAI to see how much code I can produce. This is what I am most excited about as engineer.
In my background, if I do AI research to go to DeepSeek or TikTok, very realistic, maybe need more time. If I start with AI software and agents, do some compiler knowledge to analyze program, provide context. Recently, I also did some work using Spring filter to log all requirements HTTP requires and write test cases—like five minutes I can generate 100 test cases. It’s a little improvement. Some prompt engineering, JSON truncation for context limits. What’s your point for me now? If I do AI model research, I just have the 4070 GPU. Maybe I can get two GPU? How many GPU do you have for training? Do you have 10 very fancy, 100 GPU for larger model training?
B: I don’t do training myself. I mainly work on systems; researchers do the training. I don’t have access to GPUs; I mainly use them for testing, which doesn’t require a lot of GPUs.
A: Testing the idea is just one GPU okay, also in cloud. You use the internet, or also Google Cloud as one.
B: I use both.
A: Both, and then. Working on the environment, need to test too.
B: Yeah.
Open Source Development and AutoKeras
A: So for me, like foods, foods like engineering, major 10 years full stack, just one year little machine learning, the disappearance. What do you suggest me do next five years?
B: Entering AI is definitely enough. Anything in AI tech stack—application, hardware—doing something related to AI is important because job market clear trend: companies are firing people working in other things and hiring in AI. Definitely work something related to AI. But getting into this field is hard; starting a career here is hard. I suggest start with applications because it requires defects Ortiz in hardware compiler frameworks or modeling or mathematics. Just understand the application, customer requirements, knowledge on large language models. That would be good enough, then become an expert. Then you can switch between roles to choose one that works best. Applications, for example agents, is a good thing to start with.
A: Yeah, and also analyzing the cloud code, how they do. Two years goal: how do they do? But still need somewhat wise reason to use them. How many fires added one time? What can they do? How save tokens? How background website should default which scenarios, which models have long context threshold they are setting. Now how threshold tokens, hit long context.
B: Yes, the best would be if you can build a real-world application with real users. That would be good proof for your expertise.
A: Okay. AutoKeras, I see several good things you said: using open source library, maintenance, did very good library open source. It’s one advantage, very benefit for engineer. I see several you are sharing. What’s the very big lesson about AutoKeras? For general audience, engineers who follow you, you are one, it’s very bit, you are the type of you is that order, right?
B: Yes.
A: What’s the beat license? What’s the three the O5, the very big lessons for you to do that several years? You learned deeply about several things. But sorry to be honest, I think TensorFlow generally lost to PyTorch. AutoKeras built on TensorFlow also impacted by ecosystem. How do you think about it? How you learn or think you can do better if go back five years ago.
B: A few things I learned: how to build successful open source project. First, need to start early; be the first or second to build the project. If someone already built it… Second, innovations don’t matter as much as usability. Don’t try to… I was PhD student, tried to publish research papers, but it slowed me down building a delightful project. Always prioritize usability over innovative points. You also need to prepare for changing, refactors, rewrite when clearer about user requirements. Never afraid to write from scratch.
That’s what I did with Keras; wrote from scratch upon TensorFlow 2.0. For TensorFlow losing market to PyTorch, I don’t think AutoKeras impacted a lot because people don’t care about AutoML much anymore. That’s why project didn’t grow larger. Not because of cancer. I don’t think anything I would do better. Main thing: reduce research side, pre-organize usability. This project helped me prove I am a good PhD student among peers. Solid proof; not many PhD students have that. Gave me big advantage for job application. But it didn’t give me other things to help further in career. Whatever I need now to advance career is different from past.
A: I want to share: the principal engineer Yin Wang at F5 company. Have you heard of F5? F5 is a global security company, known for solutions like Global 360. They acquired a company that specialized in static analysis, which was sold to F5. Their focus is on compiler and stack analysis for code to ensure security for clients in sectors like aerospace or military. F5 is a public company. This engineer is now working in their Tokyo office as a principal engineer. His point about AutoML is that it aims to simplify machine learning by reducing the need for manual annotations in frameworks like PyTorch or TensorFlow. Instead of manually writing operations for matrices, vectors, or tensors, AutoML tries to automate that process. However, it often doesn’t work as expected because configuring AutoML itself requires a lot of effort. So, one still has to do much of the work manually.
He also worked at Intel previously and pointed out that this solution isn’t ideal. Basically, AutoML struggles to deliver on its promise. PyTorch, on the other hand, acts like a compiler for machine learning, defining functions for operations and computing gradients like gradient descent. So, while PyTorch serves as a compiler for ML operations, tools like AutoKeras or other automated systems often fall short because full automation is challenging, and manual fine-tuning becomes necessary, right?
B: Depends on applications. In simpler situations, AutoML will work. Probably what he said is more advanced cases. AutoML wouldn’t work for everything, but definitely some simple cases that are not in color.
A: But AutoML couldn’t work in Transformer, GPT Transformers such complicated, couldn’t do that.
B: Transformer is quite simple; doesn’t have lot hyperparameters compared to other architectures. Reason we didn’t do that is mainly financial. Giving AutoML system more compute to find better solution: save cost for hiring people, keep people focusing on important things. But now bottleneck is not people.
Six-seven years ago, cost following act was people—machine learning engineers expensive, so optimize productivity for compute to run automatically. Now cost bottleneck changed from people to compute—used up all computing to train one missionary model. Cost bottleneck is compute; spent all money on computers, cost for people becomes negligible. Maybe exaggerating, so they would spend more people, manual work to save computers. This switch between computer and people made lot people not want apply AutoML to transformers.
A: Okay, got you. It’s like Firebase thing: very easy, cloud computing easy, don’t need write backend database, server. You host all to save bot, but they have limit. Use extraction more higher limits, also expression introduce convenience but limitation.
B: Cost is more compute for convenience.
A: I see OpenAI, those people under one million USD salary high, but compared to model training cost, negligible, just one or two percent.
B: Okay.
A: Cool. I seen, got, you taught me so much. I am grateful.
B: No problem. Let me emphasize: most important takeaway is proceed with caution. High requirement for background and degrees for PhD or master’s application may not suit you well. If you want to go this path rather than transfer inside company to go to U.S., degree here is definitely harder path for you.
A: Thanks very much. I also have friend: go to Hong Kong work several years, worked in BlackRock, do visualization, then Hong Kong series, then go to U.S.
B: This is definitely better path for you. Maybe avoid taking all those courses not help much in future.
A: Thank you for the information. Final question—what do you want from this project? I’ll do the transcription. If you’d like, you can share your blog or ideas, maybe something about “Intelligence Bandwidth” or life changes after living in the U.S. for 10 years.
B: Nothing in mind right now. I’ll let you know later.
A: Okay, then we can finish the call. I’d like to keep the blog updated—if there’s anything you don’t want included, I’ll remove it before sharing. You can review it first.
B: Sounds good. I’ll also send you my part of the recording using voice memo.
A: Perfect, thank you.
B: Nice talking with you today. Thanks again. Bye!
A: Bye!