Ray (Jui-Tse) Hung

Ray (Jui-Tse) Hung

SWE Intern @ Databricks, Scale AI, Meta, Amazon | NASA | On Deck Fellow | CS @ Georgia Tech

How To Reduce Cold Start Times For LLM Inference

less than 1 minute read

Published:

Reducing LLM endpoint cold start time is the last project I worked during my internship at Scale AI. And we had a blog post about the optimization we did to reduce LLM endpoint cold start time: How To Reduce Cold Start Times For LLM Inference

You May Also Enjoy

Internalizing Python bisect_right() and bisect_left()

6 minute read

Published:

Binary search is a popular topic in coding interviews. I was struggling with finding the right template for implementing binary search as it’s annoying to get the equal sign or less than equal sign correctly in the right place. Eventually I decided to use a template similar to bisect_right() and bisect_left() in Python bisect library.

Transfer from De Anza To Georgia Tech 🐝

11 minute read

Published:

In February 2023, the president of ACM Club at De Anza College invited me to speak at an out-of-state university transfer panel. There were many amazing panelists, but too little time for us to share everything. So I decided to put my thoughts and experiences here 🙂