Sid Banerjee

Musician & engineer working on audio ML, DSP, and interactive music systems.

EECS @ UC Berkeley · Class of 2027

Sid Banerjee

I'm a senior at UC Berkeley studying Electrical Engineering & Computer Science, working at the intersection of digital signal processing, music information retrieval, and machine learning.

I'm excited about using data and machine learning as a means of “crunching musical data”: revealing hidden structures in music and powering human-centric composition tools. I conduct research at CNMAT and BAIR, and teach recitations for ELENG 66 (Signal Processing & Applied Linear Algebra).

I also make music and art: synthesis instruments in Max, audio-reactive visuals in TouchDesigner, and productions in Ableton that blend electronic/hyperpop, cloud rap, and jazz.

Contact: sidbanerjee[at]berkeley.edu

Research

Evaluating the Jazz Transformer: Surprise, Risk, and Human Practice

2026

Measuring risk-taking in the Jazz Transformer's generated solos with Markov judges calibrated on human improvisation: how many risks the model takes, whether its continuations make them sound intentional, and how well they fit the chords.

Perceptually-Informed Matching Pursuit for Computer Assisted Orchestration

2026

Implementing a perceptually-weighted feature space (neurogram) and psychoacoustically-motivated audio distance metric in Matching Pursuit for Computer Assisted Orchestration.

Target 1

Orchestration 1

Target 2

Orchestration 2

Conversational Behavior Perception Foundation Model With Unified Graph-of-Thought Model

arXiv:2512.21706 · 2025

Shuchang PAN, Siddharth Banerjee, Dhruv Hebbar, Siddhant Patel, Akshaj Gupta, Kan Jen Cheng, Hanjo Kim, Zeyi Austin Li, Guan-Ting Lin, Cheol Jun Cho, Martin Q. Ma, Tingle Li, Xiaoyu Shi, Jiachen Lian, Gopala Anumanchipalli

A conceptual reframing of full-duplex interaction from next-token prediction to next-behavior reasoning, arguing that modeling the causal chain from intent to action is critical for natural dialogue.

Selected Creative Work & Projects

Model:Orchestra (Live Performance)

Max/MSP · 2026

Ambient pads derived granularly from the sound of an orchestra tuning, with parameters to control repitching, voice mixing, chord quality, and final filter. Generative piano powered by randomized rhythm and pitch controls.

2017SWAG

Ableton · 2026

Hyperpop/electroclash music.

Cross Synthesis Max Plugin

Max/MSP · 2026

Implementation of cross synthesis (spectral processing) via pfft~. The technique used is whitening the carrier's spectrum before multiplying (i.e. applying) the modulator's spectrum.

Pixl.FM VFX for Live Performance

TouchDesigner · 2026

Per-song vfx for artist Pixl.FM's live show at Thorsen House [11/7/2026]; keybind controls switch between songs and B-roll, and toggle in-song effects.

Death, Shoegaze, and Cloud Rap

Ableton · 2024

One of my favorite beats I produced.

Modal Improvisation (Instrument)

Max/MSP · 2024

Real-time MIDI functionality for a performance-ready instrument, built entirely in Max. Synth pad created from scratch using multichannel harmonics and versatile comb filtering.

Other

  • 3.3x Gain Audio Amplifier [2022]: 5.5x gain amplifier (2.4% THD) with a filter, gain, and output stage.
  • Chord Flow [2021]: An algorithm in Python with music21 to process chord changes given a MusicXML file and analyze them, structurally, for real-time practice.

Experience

JAN 2025 — PRESENT

CNMAT @ UC Berkeley

ML Researcher

Researching better evaluations for generative compositions, mainly in contextualized surprise/entropy for quantifying justified risk and creativity. Previously integrated perceptually-informed models into computer-assisted orchestration (neurograms, non-linear filters, SSIM), eliding onset segmentation and optimizing over a perceptually aware latent space.

JULY 2025 — PRESENT

Berkeley Artificial Intelligence Research (BAIR)

ML Researcher

[Speech Group] Working on full-duplex conversational behavior reasoning through predictive Transformer models and causal inference within a Graph of Thought. Most recently, I worked on an agent-agent orchestration pipeline for providing writing critique.

JAN 2026 — AUG 2026

Apple

Machine Learning Research Intern

[Acoustics HWE, Applied ML] Perceptually-informed evaluation suite for speech/music enhancement signal chains; particularly focused on non-intrusive metrics for real data and engineering predictive models for identifying aural artifacts correlated with audio quality.

MAY 2025 — AUG 2025

Amazon

SDE Intern

Designed and deployed Cerberus, an AI agent that aggregates and analyzes service performance metrics for global AdTech systems. Serves 80+ program-region combinations and is scaled to operate 24/7.

MAY 2024 — AUG 2024

Tona AI

SWE Intern

Full-stack developer for a startup integrating AI into the music production process. Cool work involving designing frontend UI/UX (React, Next.JS), deploying backend ML models (PyTorch, LarsNet, Docker), and managing databases (AWS s3, Supabase).

JUN 2021 — AUG 2022

Media Arts & Technology Program @ UCSB

Research Intern

Cognitive-science informed mappings from music to sculpture in pursuit of meaningful synaesthetic artwork (1) exploring deep reinforcement learning as an experience in extended reality i.e. live ML training in AR/VR (2).

Relevant coursework: Signals & Systems, Probability & Random Processes, Sound & Music Computing, Machine Learning, Optimization Models in Engineering, Designing Information Devices & Systems, Data Structures & Algorithms, Efficient Algorithms & Intractable Problems

Writing