2023年9月

Hello Dr. Yvoone,

I have decided not to enroll in the Master Thesis course this fall semester. It's not a decision I've made willingly, but rather a necessary compromise: If I were a Canadian citizen or permanent resident, I would certainly take it. But, if I take the course but fail to publish the paper, I will lose my PGWP. This is a substantial risk that I believe no international student would willingly take. Again, I am stubborn but not stupid. getting a paper published this semester is a mission impossible even with the help from the WPI student I introduced to you earlier.

However, my desire to continue to be your disciple remains strong. I still want to work on the thesis and produce results (dreamily, on OSDI or PODC) that demonstrate my potential as a strong Ph.D. candidate in the field of Distributed Systems. I understand this may seem to be an imposter: if my passion for the field is genuine, why should the outcome matter? But I won't fall into this "you are not a real fan" prosecution -- I refuse to be drawn into the debate of authenticity. I am genuinely passionate about this field, and I want both: a deep understanding and perceivable results.

Therefore, I've decided to change the game plan for my graduation thesis (possibly an after-graduation thesis). I plan to work on this paper outside of an official course, yet I still wish for you to be my advisor, albeit unofficially. My only request is that I would like to schedule one-on-one meetings with you to discuss my ideas and approach to the thesis. I need a community. I don't want to fight alone. I assure you that I will have a prepared agenda for each meeting to respect and make the most of your valuable time.

After conducting a recent literature review, I've found that I'm not particularly interested in HPC. To me, it's essentially a distributed system with heterogeneous hardware, and the truly interesting part lies in the scheduler, which I don't think I can get permission to do experiments in the Discovery cluster. I will still maintain my focus on Distributed Systems, particularly in the context of Machine Learning tasks. A rough idea in my head is a comparative study of the performance of the same Machine Learning task executed on Ray and Spark, with a focus on dataflow.

Why and what's Ray? Let me continue my AWS log dive story with you. It makes my internship the second most fascinating that ever happened after I came to Canada (I already shared with you the most fascinating one with love and tears).

So for the 12 PetaBytes (not 16, at that time we got the number wrong) log dive, I was required to come up with a design doc within a single day. And I made it, using the MapReduce paradigm -- yes, the very one we discussed in our first class, where we officially met for the first time. I also asked Peter's opinions on using the Step Functions as the coordinator (master) for the whole process. Unfortunately, he didn't have the time to review my design documentation. The core concept was to have an Extract, Transform, Load (ETL) job, powered by a Spark Executor with 128 GB RAM and 32 vCPUs, handle one day's worth of data (roughly 3 TB). This would represent one map task. And the 12 PetaBytes data, according to my experiments, would spend Amazon ~8 million dollars.

To be honest, upon starting the design, I intuitively felt that a Spark Executor shouldn't be one map worker for such a task -- it's just a grep task, and The data were stored in JSON (that's why the S3 bucket took 12 petabytes and costs Amazon 300k per month) so there's no "dataframe" and all we need to do is to do a String Match, the KMP one. But the bottleneck for this task is S3's IO and it seems that there's no other tool I could use except Spark to run the grep efficiently to exploit the S3's throughput limit. They want results badly. So speed really matters here. No time for me to write the code from scratch that controls a bunch of EC2 instances, even using Elastic MapReduce.

Luckily, one day I got some help from a nice guy Zach. He is a Senior Big Data Architect and, on hearing my task. He relentlessly recommended I use Ray which has just been published in AWS Glue (the product hub for serverless data analysis and computation, Spark (Glue ETL) is also there).

"How could I have overlooked Ray?" I questioned myself during the meeting. After all, it's the framework behind Alpa. What a serendipitous connection! Ray is also another framework that handles intensive ML dataflow, especially Reinforcement Learning, invented by RISElab, the one that invented Spark. Ion Stoica is the Oppenheimer behind the two projects. In my real-world experience with Ray, it almost turns programming for distributed systems as easy as parallel programming. All the nuances in consistency, availability, and partition tolerance are hidden from users. They only need to write Python code. (I feel sorry it's not your favorite Rust.)

Zach also wrote me a demo for such a log dive. After I modified it according to our needs and did some fine-tuning, the cost for a scan was estimated at merely 500 dollars.

So from 8 million dollars to 500 dollars, that's a BIG win. But, just like my love story, reality is not a fairytale. Amazon ultimately decided to use an existing tool to do the scan, partially because Ray is not available in some of the regions, but mostly because they want the result badly right now and cannot take the risks.

As for me, my internship was extended until December, and I was assigned another project. Despite having the design documentation and a prototype using Ray ready, I have to put them on hold, most likely eternally.

Very sincerely yours,

Liao-Liao

Life's all about timing. You gotta know when to pull the trigger, when to go in for the kiss, and—most of all—when to make for the exit.