PITCH PERFECT

A hardware-accelerated real-time phase vocoder for pitch scaling, built on the Intel/Altera Cyclone V DE1-SOC FPGA

SKILLS

• Embedded System Design
• Designing hardware circuitry to perform complex, multi-step mathematical algorithms
• SystemVerilog hardware description
• Intel Quartus and QSys Platform Designer
• Interfacing with prebuilt hardware components
• Circuitry timing
• Fixed-point mathematical operations in hardware
• C Programming
• Linux Kernel hacking to allow software to interface with hardware
• Embedded system simulation in C
• Digital Signal Processing
• Fast Fourier Transform
• Phase Vocoder Algorithm

DESCRIPTION

In my last semester of college, I decided to take an Electrical Engineering capstone class, Embedded Systems, despite not actually being an Electrical Engineering major. I had really enjoyed an EE class that I'd taken in which we'd built up a working CPU able to execute assembled machine code from basic logic gates, and I was curious about how other hardware systems worked. I had also taken Operating Systems the semester before and was curious about the physical connections between hardware peripherals and software. I hadn't taken the recommended prerequisite classes for Embedded Systems, but I was curious enough about its content that I figured I'd be able to catch up where I needed. The class ended up being a great experience. I learned all about the most important components of modern embedded systems (memory, networking, USB, video, etc.), including their histories and how they're commonly implemented now.

The course also included a semester-long group project in which we got to work closely with our professor to design and then implement a project with both hardware and software components on an FPGA. My group of 3 decided to build a pitch shifter/scaler that would take in audio from our FPGA's 3.5mm audio input line and scale it's pitch entirely in hardware using a custom circuit we designed, then play back the audio to the board's output line, with the level of pitch scaling being controlled via software. To do so, we first started by researching different pitch scaling algorithms and implementing them in Python to gain a thorough understanding of the workings of each and decide which sounded the best for our use case. We landed on a version of the Phase Vocoder algorithm with transient detection for improved vertical coherence. We then implemented the algorithm in C, simulating intended hardware implementation by using fixed-size memory buffers, polar coordinate conversion of transformed frequency-domain signals to remove the need for trigonometric hardware calculations, and a real-time streaming interface, among others features. Once we were satisfied with the sound of our C implementation, we got to work designing circuits for our board, which was able to reconfigure itself to test our specified circuitry. You can find the code for Pitch Perfect here and our final presentation slides here.

SPYGLASS

An application for monitoring and validating SCTE-35 cues in MPEG-DASH and HLS video streams to ensure proper advertisement placement

SKILLS

• Digital video
• OTT video streaming
• ABR streaming (HLS and MPEG-DASH protocols)
• Low-level video encoding and compression (YCbCr and chroma subsampling, motion estimation, DCT, video codecs, etc.)
• AWS Media Content Distribution Suite (MediaLive, MediaStore, MediaConnect, etc.)
• SCTE-35 protocol
• Dynamic Ad Insertion
• AWS DynamoDB SDK and database design
• Docker and Kubernetes
• GraphQL
• JavaScript and React
• Python and Flask

DESCRIPTION

For my second big project as a Software Engineer Intern at Paramount, I was tasked with creating an application for ensuring that SCTE-35 cue messages are properly inserted into HLS and MPEG-DASH video streams. All of my team's apps have a nautical-themed title, so we decided to call this Spyglass, since that's what a pirate uses to look ahead see what's approaching before they reach it. The application started as a stripped-down version of another app that my team had built with a similar architecture, containing only the bare essentials to deploy with Docker and our CI/CD framework. I began by creating a Python/Flask backend to take in and parse encoded SCTE messages. To determine which sections of cue messages were necessary to consider given the project's goal of improving advertisement placement, I read (skimmed) through the 100+ page SCTE-35 standard. I then wrote code to verify and log the validity of incoming cues in an AWS DynamoDB, which required some thinking about how to design the data structure containing this information to allow for efficient lookup, and unit tests to make the application easier for my team to maintain after my internship ends. Because I was able to finish this project a couple weeks before my internship ended, I had the opportunity to integrate it with my team's main frontend monitoring system, a Next.js app using GraphQL. To do so, I created GraphQL queries to retrieve networks' logged cues, then displayed them in a React table, and added the ability for users to filter by date, time, and substring match.

This project offered a ton of learning opportunities, both relating to the tools used to create it, and to digital video in general. For example, while debugging, I learned all about Docker and Kubernetes, which my team used to deploy all of our interconnected apps. I also wanted to take advantage of my time being on a team that focused on video distribution and delivery for the summer and learn all that I could about digital video. My great mentors shared lots of resources with me, and I spent a lot of time reading about and discussing all aspects of digital video, from how cameras work to encoding and compression to distribution to digital displays.

FOGHORN API

A tool for Software Engineers at Paramount to easily log messages to a unified location and recieve Slack alerts when sections of code are executed

SKILLS

• A variety of AWS Cloud Services (Lambda, API Gateway, IAM, CloudWatch, CloudFormation)
• Designing and creating a secure access management system
• Deploying Infrastructure as Code with the Serverless Framework
• GitHub Workflows for automatic deployment to development, QA, and production environments
• External API integration (Slack API)
• JavaScript
• Python

DESCRIPTION

For my first project as a Software Engineer Intern at Paramount, I was tasked with creating an eagerly-requested API that would allow developers to easily log messages to a unified location when sections of their code were run. To do this, I created an API using AWS Lambda functions that would interact with CloudWatch to log messages. I then added functionality allowing engineers to send these messages to Slack by integrating with Slack's API. Because the rest of my team's tools were named to followed a nautical theme and this can be used for sending alerting notifications, we named this Foghorn. To deploy Foghorn, I learned how to use the Serverless Framework, which made it simple to deploy development and production versions without directly interacting with the AWS console.

SHEEPSHEAD AI

During our Spring Break, my friend Adam and I built a simulator of our favorite card game, Sheepshead, then trained an AI to play it using Monte Carlo Simulation and Deep Reinforcement Learning

SKILLS

• AI game playing
• Deep Reinforcement Learning
• Monte Carlo Simulation
• TensorFlow
• Python

DESCRIPTION

Whenever I'm home from college, my friends and I get together to play Sheepshead, a strategy- and memory-based card game. On one of these nights, someone brought up the idea of building a website where we would be able to play together while living at colleges over 1,000 miles apart, which spawned the idea of trying to build a bot that could beat us. During Spring Break 2023, my friend Adam and I decided to tackle the challenge. We started by mapping out on paper how we would model the game, then built that model in Python. The model included a player class, who would be able to take their turn by selecting from a list of allowed plays. Then, we were able to train the AI to play by simulating millions of games with other bot players. It works by evalutating all allowed moves and deciding after which one it has the highest chance of winning, based on it's training data. Using deep reinforcement learning, it is even able to generalize and make predictions in situations that it hasn't seen before. After a certain number of training episodes, we update the other bots the AI is simulated against to match the AI's current decision-making, increasing the strength of it's competition and forcing the bot to make smarter plays. You can find our code here.

IMAGE CAPTIONER

For the final project for my Natural Language Processing class, I built an image caption generator

SKILLS

• Computer Vision with CNNs
• Stacked/Deep Bidirectional Recurrent Neural Networks
• Generative Language Models (GPT, Transformers)
• GCP Deep Learning VM
• PyTorch
• Python

DESCRIPTION

For the final project for my Natural Language Processing class, I built an image caption generator. This project combined many of the concepts we'd learned about in the class with ideas from Computer Vision. It began with a pre-trained CNN that constructed a vector encoding representation of images. These were then fed as inputs into a generative language model of stacked BiRNNs, which at each step output the probabilities of each word in the training dictionary being the next word in the caption. I then implemented a Beam Search algorithm to output the most likely caption.

NEURAL NETWORK

To gain a better understanding of deep learning, I built a neural network from scratch

SKILLS

• Neural networks and deep learning
• Numpy and Pandas
• Other machine learning methods (k-nearest neighbors, k-means, decision trees, support vector machines)
• Python

DESCRIPTION

After my tech strategy internship ended, I wanted to get a deeper understanding of deep learning, specifically neural networks and how they work, so I built one from scratch using Python. My neural net uses a sigmoid activation function for an added computational challenge and manually calculates and applies partial derivatives for backpropagation. It can also scale to contain any number of hidden layers, each of any size, that it is initialized with, and accepts both batches and individual inputs. You can find the code for this project here.

SONAR INTERNSHIP

After my first year of college, I was a Product Manager and Software Engineer intern at Sonar, a digital health startup

SKILLS

• Product Management
• Competitive analysis
• Web scraping
• Figma
• Numpy, Pandas, Django
• Python

DESCRIPTION

As a Product Manager intern at Sonar, I worked under the guidance of a team of medical experts from Columbia's Medical School to create specifications for some of the app's core features, which provide users with detailed information about specific aspects of their health. I then worked cross-functionally to guide design and engineering teams in building out these features and others. As a backend software engineer, I was able to build out some of the features I defined while learning about the software development process. Since Sonar's launch on the app store, it's been great to see that features of the app I helped build are now adding value to people's lives, especially as a user of the app myself.