The three of us had a great first day at Laber Labs!
In the morning, we started by working on a quiz that compared a statistician to an inanimate object, with our statistician being David Blackwell and we decided on a spatula being our inanimate object, and we did research to come up with some questions for both.
In the afternoon, we created levels for a game called Zombies on Treadmills with Danny Schmidt and Sasha Chirova (two game designers at Laber Labs) and experimented with the different map sizes for levels. Some of my level designs include Pikachu and E.T. for the Atari.
Starting tomorrow (Wednesday), I’ll be beginning my work experience period at Laber Labs, led by Dr. Eric Laber. According to their website, Laber Labs is a “statistics lab dedicated to the development of practical and mathematically rigorous methodology for data-driven decision making.” I believe being able to analyze data is a skill that can prove helpful in multiple career choices, so I’m particularly excited for this work experience and to see where it takes me!
– Maddie L
p.s: Since we’ve yet to start and haven’t taken any photos yet, please take this stick figure of me waving
I’m extraordinarily grateful to Dr. Eric Laber and Laber Labs for agreeing to host us for our Work Experience Program. According to their website, Laber Labs is “a statistics lab dedicated to the development of practical and mathematically rigorous methodology for data-driven decision making”. Laber Labs has been working on many projects, including precision medicine to tailor treatment decisions based on the characteristics of each individual patient, spatio-temporal reinforcement learning to inform the management of emerging and persistent infectious diseases, and adversarial decision making to understand the decision making process of two agents whose goals are either imperfectly aligned or completely at odds. I’m excited to see what awaits me in this program.
Today we started with reviewing our findings of yesterday’s Monty Hall problem. We found that in our datasets switching resulted in a win more often than staying. The reasoning was the chance of winning for staying was 1/total number of choices while switching was total number of choice – 1/ total number of choices.
Next we examined a problem about marshmallows introducing us to statistics. We had scenarios where they were different colors and sizes and were tasked with finding the proportions between color/size and the total population. Finally we were asked to find the total amount of marshmallows and found real world applications of this problem, for example estimating the number of a specific animal in their environment.
For the remainder of the day we will be examining the meaning of r^2 in statistics, fitting a linear regression in python, and proving R2(y~x1) + R2(y~x2) >= R2(y~x1+x2).
Today, we were introduced to Dr. Laber, a professor at Duke and the head of Laber Labs, a research lab that researches different questions, mainly related to medicine, and uses methods like statistics and reinforcement learning to solve them.
Pictured below is an outreach project where Reinforcement Learning was used to “teach” a computer to play the Laser Cat game.
A Demonstration of Q Learning in the Laser Cat Game
For the rest of the day, we’ll be working on simulating something called the Monty Hall Problem. Basically, there are 3 doors. Two have something you don’t want and the third has something you do want. You choose one door and another one is eliminated, leaving you to choose to either open your current door or switch to the other remaining one. What we’re doing is building a dataset to see how often you win when you switch doors and if there is a correlation between switching doors and winning.
Believe it or not, there was no stats lesson today. We sat down with Dr. Laber and had an hour-long discussion about his path to becoming a professor. He grew up in Michigan, exhausted the math courses at a tiny local college, and decided to attend UCLA his sophomore year. Beginning a program for students who had completed all undergraduate math classes, he found statistics as his way of applying the abstract concepts. Overall, our time with him was very enjoyable, as his easygoing attitude and quirky humor always kept things interesting. We gave him our gifts as thank-yous for the time he spent with us, and headed to the BOM to meet with this man Dr. Laber only referred to as “Rob” from CAA.
Turns out Rob actually had worked at the CIA, not CAA, for about 15 years after getting a degree in chemistry and working in the NASA virtual laboratories and teaching at Yale. So, quite the guy. He’d been to countless countries and had so many interesting stories to tell. His doctorate and master’s degrees had both been fully funded by NASA and a couple other major corporations. It was a really unique experience meeting someone as accomplished and well-rounded as Rob, who I felt had been given quite an unassuming description by Dr. Laber. Before we knew it, our meeting was over and Suki and I were saying goodbye to the graduate students we had been working with for the last couple weeks. Thankfully, we agreed to keep in touch with Rob, Lisa (the graphic designer), and Allison (the programmer set to work at Apple starting September).
All in all, the experience was something I hadn’t really expected. While the work was sometimes tedious, and the lectures sometimes terrifying, the knowledge I’ve gained and the people I’ve met made it completely worthwhile. Being on a college campus was also a fun experience, being able to hang out at the Tally Student Union (which I would highly recommend) and walking through all the buildings from SAS to the BOM made NCSU feel pretty homely for the past 8 days. I’m glad to have been a part of it.
Suki and I in our Laber Labs shirts with Dr. Laber himself
I can’t believe it’s the second-to-last day already. We finished our presentation on multistep regimes and transitioned into a more qualitive approach. Dr. Laber spoke of various models to be the most efficient and most effective options, acknowledging each’s strengths and weakness. For example, he proposed that patients themselves could select treatment preferences; however, health literacy is notoriously low, as the common person won’t understand these complex patient regimes with so many variables and potential outcomes.
During the second part of the day, Suki and I continued to create levels for the Boredoom game. We began incorporating more complicated turning sequences (now with some trees spinning two times per turn or changing its direction from clockwise to counterclockwise). It was just as monotonous, if not more, than yesterday’s work. Sometimes the most difficult part of the level would be naming it, as we often disagreed. We joked that after an entire year of being lab partners in AP chemistry, through all the stressful lab reports and exams and notebooks, this game was the biggest test of our partnership yet. We had finished 22 boards total before Dr. Laber asked us to write a text-based algorithm to describe a sequence of actions that could be programmed into a computer to solve any Boredoom level. It was difficult to match the level of specificity Dr. Laber was looking for, as we had to recognize the possible moves 1 move away from the end square, to 2 moves away, and so on until it had “bloomed” out to reach the starting square.
The various 3D-printed Boredoom piecesAll the lovely diagrams we filled out for hours on end, with each tree labeled and the correct pathway numbered
And on a non-work-related note, Suki and I went to Smash Waffles today on Hillsborough street, who had the best waffles I’ve ever had. Here’s an image of our incredibly nutritious lunch, including flavors such as chocolate chip cookie, maple bacon, and salted caramel:
Last day of stats lessons!? Maybe? At least, this’ll be the last time I try explaining one of the algorithms. Today’s focus was on multistage regimes (quite the party) and concerned the new variable H for history of treatments a patient receives. Here’s the equation:
Remember:
Y = outcome
X = patient characteristics
Q = quality, or type of function
A = treatment
A bar = sequence of A
E = expected outcome
∏ = product operator
t = time
T = subscript just to indicate transposing of data to fit the code
Dr. Laber explained how this function defines a potential outcome under some regime [Y*(pi)] as a summation of all “a” values within the function defined by the sequence of treatments received. Then this is multiplied by the product operator acting from t = 1 where the regime at some t value by the sequence of “a” treatments with the history of such treatments before the trial took place. Ultimately, the entire right side of the equation can be negated if the sequence provides a 0 or all 1s. Woohoo! The final equation below just explains that the expected outcome of an optimal pi regime must have a greater or equal utility value that any other outcome (making it the most favorable). My parents said I’d better want to be a stats major after all this, and I’m starting to agree.
In the second part of the day, Suki and I worked in the BOM and helped design levels for a new board game – Boredoom. Based off a board game and the word “boredom” due to the tediousness of creating it, the game has a 3D printed goat trying to evade rotating trees that (in theory) shoot pine needles on a classic chess board layout. While Louie was off having the board be carved out of wood, Suki and I were planning out 12 different levels, annotating the correct move set, and transcribing it to sheets of paper. It was both arduous and exasperating to keep track of all the pieces, but we’re happy to be helping the Laber Labs team. Plus, we’ll be featured in the credits!
Second week started with a bang, and by that I mean Suki and I were given donuts halfway between our daily stats lesson. Today, Dr. Laber continued his presentation on modeling the ideas behind precision medicine. He spoke of the Horvitz-Thompson estimator that introduced an unbiased estimator under probability sampling designs:
(click the image for better resolution)
The argmax operator defines input value for maximum output, capital as the optimal regime (most favorable outcome), p as the probability value, w as the event, a as the patient treatment, and x as the correct treatment to provide to the patient. The equation just lays out the fact that the probability for an optimal “a” as defined first by x as a component of event w of components y, x, and a. It’s… rather confusing… There’s a lot of statistical jargon that’s been pretty difficult to understand at first, but it’s become (slightly?) clearer through these mini-lectures.
Dr. Laber also gave us a problem before giving us 7 minutes to solve it consisting of rolling a dice three possible times and wanting to attain the maximum score. We were able to find the correct “cut-off” numbers of {5, 4} for turns 1 and 2 before Dr. Laber stepped in and showed us through the program R that those two numbers did indeed yield the highest possible score. We then met with three very accomplished people, beginning with former department head Dr. Sastry Pantula, who’d been at several high-ranking positions (including the NSF!) and now serves as a dean at Oregon State. We only had about 10 minutes before he had to leave, but we spoke with him about his life, his work, and his daughter (who’s also a rising senior). Next was Dr. Brian Reich, whose work with dust samples later became the plotline for a CSI episode! The fungi from the dust enabled him to pinpoint the origin of the dust within 200 km through the use of many (complicated) statistical equations. The last person we spoke with was Ryan Martin, one of the most published people in the world for his age. His theoretical approach to probabilities defied all previous work with the Bayesian and Classical forms of statistics, earning him a fair share of supports and critics. All the people we spoke with were incredibly accomplished and interesting to speak with- I feel really grateful that they made the time to talk to Suki and me.
Dr. Laber and his absolutely, 100%, fully comprehensible R programming
Friday! Today, Dr. Laber’s daily stats lesson consisted of the math behind precision medicine. It was, by far, the most complicated lesson Suki and I have ever received in our lives. He went over insanely convoluted functions for sequential multiple assignment randomized trials (SMARTs) for max efficiency of data. These included Q functions (a type of trial) of the three variables {(Xi, Ai, Yi)} for certain patient characteristics that would in theory spit out the proper treatment using digits of {-1, 1} – I know, not the most intuitive of equations. Before we knew it, Dr. Laber had moved on to linear regression-based estimations like this one:
He also tied in a bit of calculus, which was thankfully something I could understand, when looking at optimal regimes (best treatment flow-path for a patient). His lessons have proven to be both interesting and wildly terrifying, especially since Suki and I are the only ones in the audience. The second part of the day was composed of working on an augmented reality (AR) 3D imaging project with Lisa, the graphic designer. AR is a special type of animation that uses real-time images, such as a video feed from a camera, and placing animated objects on top of it in the virtual world. She had us create animations using the engine Unity that would be triggered by the camera recognizing a certain object- a card with our names on it. Three hours later, we had a (somewhat) functional model in place; mine included the letters of my name peeling off the page while trees grew and shrank in the background and food spinning circles in the bottom right corner. While it was fun trying my hand at 3D animation, the navigation of so many windows and the difficulty of the program overall showed me how I could never be an animator. That stuff must be HARD. All in all, I’d say it was a successful first week at Laber Labs, and amongst all the unintelligible codes and graphs, it’s been a good time.
Dr. Laber giving his daily stats presentationUnity’s interface, plus a sneak peak at my sub-par animation skills
