More CartPole (seeds!)

Today, we wrapped up the WEP and asked any lingering questions we had and worked on an independent project for the rest of the day. Naturally, I decided to try to figure out the issue with CartPole. The most likely issue is a min and max switched up but I feel like I’ve checked them all so I decided to try something else – seeding the environment. Basically, every time the environment is reset at the beginning of each episode that the agent plays, the initial condition is randomized. However, by seeding it, the environment is reset to the same condition every time. Minecraft’s world generation operates on a similar base concept. If no seed is given, then the world is generated randomly with a random seed each game but if a seed is given, then the world generated will be constant regardless of the number of times the seed is used. By seeding the environment, I’ll be able to tell if the issue is in the agent being too dumb to get a general ‘understanding’ of how to win in randomly generated environments but smart enough to figure out a solution after playing a single environment enough or if the issue is actually just something critically wrong with my code. It’s been running for a bit and I’m feeling a bit optimistic because the score seems to be consistently around 20 points and slowly rising, although there’s still the chance it’ll slowly get down to 8-10 points again.

An image of my code and the agent’s reward over many episodes.

 

Update: After a bit more training, the reward is approaching like 10 points again so I think the issue is in my code rather than the seeding/generation of the environment. Debug time!

The Final Model

I spent this morning adding a couple more sliders and switches to our model as extra features for the user to play around with. In the afternoon, Ethan and I met again to combine our code one last time for our final model. Seeing the final model was really cool as it really showed how much we’ve done these past two weeks. Here’s a before and after:

Before:

After:

Day 8: overall reflection

I think my main takeaway for this experience is the importance of marketing and branding. I’ve realized that no matter how good you execute things, it won’t be impactful unless people know about it. It’s important as a business to have an online presence especially now that so many things are virtual. Overall, I’ve learned a lot about marketing, Homestead Sage, and UVC technology through WEP.

(I forgot to take a picture of our final meeting)

Day 8: The Finale

Today we presented our final presentation to Mr. Self, and it went really well. It felt really good and went way better than I thought it would (not that I had low expectations or anything). It was actually kind of casual, especially since sometimes he would pose a question and then we would have a small discussion before getting back to the presentation. This made the whole experience less nerve-wracking. I’m really proud of what my group and I accomplished these past 8 days, and the presentation to Mr. Self was proof of all our hard work. It’s kind of sad to think that this is the last day, but it really was a great experience and I had fun. Also, spending our last day together in person was a nice way to finish things.

Day 8 – Final Presentation!

Today we finished up our work experience by presenting our PowerPoint to Mr. Self at his office building. While heading up to the conference room, we ran into the other group working with Vector Textiles (we had to kick them out of the room). We even met Mr. Self’s dog (very cute and also very shy). The presentation went well, Mr. Self had a lot of interesting questions. He seemed to like the information though! If we had more than 8 days I feel like we definitely could’ve dug deeper and done more research on some of the questions he brought up. This work experience has been great, and I had a lot of fun researching the effects of permethrin in sub-Saharan Africa. I learned a lot of new and interesting things. I’m really grateful that I got to work with Mr. Self and his company Vector Textiles!

Abigail!!

After lunch, we met virtually with Abigail Quesinberry, FWV’s Social Media and Influencer Marketing Associate Vice President. We learned about boosting posts on social media, which entails paying companies like Facebook and Instagram to promote the given ad so more followers view it. We also gained insight into the extent of the social media ideas included in a Request for Proposal. Abigail explained how the goal is to impress the potential client while not giving them enough information to execute the idea without FWV. In addition, we inquired about working with influencers, and Abigail outlined the benefits and difficulties that come with influencer marketing. Surprisingly enough, the best influencers to work with have 50-200k followers because they are less expensive, have followers that are more engaged, and more often go above and beyond what is expected from them. All in all, our last day in the office was successfully bittersweet!

Moments after our Zoom meeting with Abigail
Skip to toolbar