A Bay Area Rapper and Software Engineer Made an AI Album in 24 Hours

Stop me if you’ve heard this one before: an East Oakland rapper and Berkeley software engineer fly to New York City to record an album at a DIY studio using beta open-sourced AI technology, then return to the Bay Area to finish it. Oh, and they do it all on a whim, within a total of 24 hours.

That’s the universe (or metaverse?) we live in today — in which community and creativity are inescapably intertwined with technologies we don’t completely understand, but are learning to maneuver in fast-moving, self-directed ways. 

It’s a reality that Nimsins and Vinay Pai (the rapper and software engineer, respectively) hoped to leverage as friends and artists in their latest collaboration, SENSORY OVERLOAD. 

Using a test version of Meta’s MusicGen (which allows users to generate music from text prompts) and Hugging Face (an open source website where emerging AI research is shared for public use), the creative duo came together for what they’re claiming is the first album to use the latest generative music production technology.

“One of my coworkers used to work on Fundamental AI Research [FAIR] at Meta. He worked on deep system stuff, not audio. He told me about [MusicGen] existing [before it went fully public],” says Pai, a UC Berkeley computer science graduate who came up with the idea and reached out to Nimsins, with whom he had previously worked on an album. (Pai also produces music under his first name, Vinay.) 

The complete project includes the album, an AI-generated music video and a two-part (human-made) documentary about the process. The goal wasn’t necessarily to make the best rap record of all time. It was more about experimenting with new advancements in AI that, at the time the project was recorded last summer, had just leaked to the public on research back channels.

The eight-track, 15-minute effort may indicate a growing trend among Bay Area musicians and adjacent hobbyists, who will likely rely more heavily on AI-augmented tools to make art. And it certainly raises questions that the artists themselves are still attempting to answer.

So you might be wondering: Is the album any good? And what is lost when human artists yield such a substantial part of their creativity to machine-learning algorithms?

Bridging tech and hip-hop in the shadows of Silicon Valley

Pai isn’t your ordinary software engineer. While studying at Cal in the mid-2010s, he began making hip-hop instrumentals in his spare moments, and met producer Versâam, who was a student athlete on campus at the time. Versâam recognized Pai’s talents and enlisted him as an intern at Emeryville’s famed The Grill Studios, where Bay Area rappers like SOB x RBE, Zaytoven, Richie Rich and so many others — including Nimsins — regularly appeared. 

After two years at The Grill, Pai ascended in the local music industry, and went on to produce commercials for major companies like Coca-Cola to air during the 2021 FIFA World Cup. However, he often felt underpaid and undervalued for his work. Once the pandemic forced the studio where he worked to temporarily close, Pai pivoted to his background in engineering, opting for a career that could afford him to continue making music on the side. The right opportunity came for him at that moment: AI development.

“I’m not going to sit here and negotiate with [music executives] about pennies when someone can give me life-changing money to work on AI,” Pai says. “That was a year before the big AI boom. I was really fortunate to be put in that position.” 

“AI is hitting a critical mass,” he continues. “The conversation I’m having with artists is about that. I feel lucky to be a bridge, being in the belly of the beast as far as knowing the technology. Let’s figure out a way [as artists] to use it and take advantage and elevate ourselves.”

On the other hand, Nimsins — who is a father and mentor to younger artists in East Oakland, and is currently studying anthropology and Spanish at San José State University — isn’t embedded in tech. Listening to his discography, you get the sense he’s more interested in philosophy and breaking bread with his people than in profit gains and building an NFT portfolio.

For Nimsins, SENSORY OVERLOAD stemmed from his relationship with Pai — who Nimsins praises for simultaneously juggling his roles as a sound engineer, talent manager and vocalist on the album — and his own belief in experimenting with the unknown. After all, isn’t that what artists do — regardless of their technological era?

“We recorded some good parts on the album, but there’s some duds, too,” Nimsins admits. “We would sit there for hella long trying to find the right beat, finding the right prompt [to type into MusicGen]. There’s a point in the documentary when you can see how everyone’s spirit changes when we find a good beat. The spirit of collaboration hit the room in a different way.”

And yet, music is also ultimately about the final output, and not just the process.

In the documentary’s first part, filmed in New York City, Pai riffs off the cuff: “The reason I play music is because it brings me joy to be in the wave. When you’re in the pocket playing [instruments] with other people, when you’re in sync with them, like when you’re freestyling or you’re in a cypher, when you’re in a band and everybody’s locked in, that’s the joy of music. At the end of the day though, the money doesn’t come from joy. The money comes from the product.”

Pai’s outlook on the subject maintains a genuine how can we learn to use this before it uses us? tone throughout. But, of course, it may not be so simple in an industry where the “product” is also becoming devalued. Access to recording equipment has exploded, effectively decentralizing the music industry with an influx of content to stream on corporate platforms that, as Pai acknowledged, pay literal pennies.

To his credit, Pai seems to understand the layered, tangled mess of it all, and isn’t operating with AI blinders on, either.

“If you want to do what music is meant to do, which is uplift the spirit, to uplift the soul, to bring communities and people together, you have to keep those traditions alive somehow,” he says.

Not completely automated… yet

“Japanese jazz.” “Black church music.” “Early 2000s Missy Elliott.” “Experimental psychedelic reggae.” “Nigerian drill.” “’90s DMX type upbeat beat.” “Lo-fi hip-hop with futuristic soul sample.”

These are all phrases that a room full of creatives churned out together while using MusicGen. The documentary reveals a loose yet meticulous process of back-and-forth deliberation between the engineering producer (Pai), the MC (Nimsins) and a variety of guest artists (friends, rappers, photographers, videographers) who rotate within the 24-hour window to add their two cents, lay down a verse, then dip.

The differences between any beat the skeleton crew approved or canned could be as nearly imperceptible as typing “drill beat with Japanese jazz sample” instead of “drill beat with experimental flute sample” into the text box. Using screenshare recordings, the documentary captures the hivemind brainstorm as each word is collectively supplied, spraying AI-generated noise in random directions before calibrating the aim with each pull of the new-phrase trigger. It’s an admittedly mesmerizing experience to witness, and one which subtly highlights a different kind of creative problem solving that goes into making an AI-assisted soundtrack.

At one point, the camera turns to Pai, who is asked about his thoughts on this emerging tech. His response is sobering, transparent and laden with complex truths: “It’s gonna make it a lot easier to express your ideas. People who don’t have the training, background or experience can get their ideas out way faster. So it’s going to empower a lot of people. But it’s definitely going to change the business.”

“If AI can make [an instrumental] in five seconds, way faster than me… it’s gonna fuck up the producer game for sure,” he adds. “But as artists you gotta adapt and find a way to create new forms of art.”

Nimsims doesn’t shy away from the contradictory elements that underpin the duo’s forward-thinking, if not opportunistic, AI-rap dabbling, either.

“I wouldn’t do it again myself,” says Nimsins. “I know hella dope producers. I like being hands on. It only worked because Vinay is someone I trust and enjoy. But I like to make my own beats, looking for samples, all that. It was something to do in the moment, like a challenge, something we should learn more about.”

On the opening track, “ALGORITHMS,” Nimsins spits with his usual word-bending subversion: “Face to the screen and screen to the face / …An algorithm with no rhythm to name / Had ideas, they beginning to fade / Was unique, now wе one and the same / With nobody to blamе, an entity without a thing we can name.” 

The project involves nine credited musical artists — among them are Indian American rappers and Puerto Rican lyricists, who references are as varied as Islamic teachings and a full verse in Spanish. Considering it was written and recorded using pirate studios on both coasts of the country, all in 24 hours, the album low-key slaps and shows no discernable signs of automated production.

Call it new school, but you can’t knock the cyber hustle. Maybe the Too $horts and E-40s of tomorrow will just have to be out-the-high-tech-trunk with their digitized game in ways previous generations didn’t have to be. Maybe AI developers will supplant record labels.

Maybe we’ll have to embrace what the future holds for tech-savvy artists, and what tech-savvy artists hold for the future.

‘SENSORY OVERLOAD’ is available on Bandcamp. All proceeds from the album go to Beats Rhymes and Life, an Oakland-based non-profit for hip-hop education.