Just like Computer Vision getting transformed after the implementation of Convolutional Neural Network in 2012, it’s time to apply Deep Learning algorithms for Music. This new way of analyzing, characterizing and generating music using Deep Neural Networks will assist (not replace!) a musician or a composer to produce unexplored sounds and styles. In addition, these algorithms with interactive interfaces will definitely enhance the creative potential of human beings through AI.
In this blog post, I have collected some resources such as papers, libraries, demos, people and teams working on applying Deep Learning for Music. Feel free to share or bookmark this page. I will keep updating this page with more resources as I learn more in this domain.
Papers
- A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
- Latent Constraints: Learning to Generate Conditionally from Unconditional Generative Models
- Learning Latent Representations of Music to Generate Interactive Musical Palettes
- Magenta.js: A JavaScript API for Augmenting Creativity with Deep Learning
- Mixed-Initiative Generation of Multi-Channel Sequential Structures
- Onsets and Frames: Dual-Objective Piano Transcription
- Hierarchical Variational Autoencoders for Music
- Learning to Create Piano Performances
- Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders
- Tuning Recurrent Neural Networks With Reinforcement Learning
- Audio Deepdream: Optimizing raw audio with convolutional networks
- Generating Music by Fine-Tuning Recurrent Neural Networks with Reinforcement Learning
- Deep Music: Towards Musical Dialogue
- A Generative Model for Distance Patterns in Music
- A Generative Model for Rhythms
- Deep Learning for Music
- Deep Learning Techniques for Music Generation - A Survey
- Generating Long Sequences with Sparse Transformers
- Enabling factorized piano music modeling and generation with the MAESTRO dataset
Libraries
Demos
- DeepDrum
- Beat Blender
- Latent Loops
- Melody Mixer
- Performance RNN
- Piano Scribe
- A.I. Duet
- The Incredible Musical Spinners from Latent Space
- Tenori-off
- NSynth: Sound Maker
- Robot Neil’s Bubble Bath
- Latent Cycles
- Neural Arpeggiator
- Neural Drum Machine
- Neural Melody Autocompletion
- Deep Roll
Blog Posts
- How we made music using Neural Networks
- How to Generate Music using a LSTM Neural Network in Keras
- Deep Learning with Tensorflow: Part 3 — Music and text generation
- How AI-generated music is changing the way hits are made
- Using Machine-Learning Methods for Musical Style Modeling
- LSTMetallica: Generation drum tracks by learning the drum tracks of 60 Metallica songs
- Piano Genie: An Intelligent Musical Interface
- A train window
- Piano Transcription in the Browser with Onsets and Frames
- Multitrack MusicVAE: Interactively Exploring Musical Styles
- Connecting with Music Through Magenta.js
- Generating Long-Term Structure in Songs and Stories
- A Recurrent Neural Network Music Generation Tutorial
- NSynth: Neural Audio Synthesis
- Making a Neural Synthesizer Instrument
- Generate your own sounds with NSynth
- Generating Classical Music with Neural Networks
- Analyzing Six Deep Learning Tools for Music Generation
Videos
- Making Music and Art Through Machine Learning - Doug Eck of Magenta
- Project Magenta: Music and Art with Machine Learning (Google I/O ‘17)
- Magenta’s AI Jam: Making Music with TensorFlow Models
- Making music using new sounds generated with machine learning
- Making music with NSynth Super
- Music with Artificial Intelligence | Andrew Huang
- PYCON UK 2017: The Robot Composer Writing Music With Machine Learning
- Music and Art Generation using Machine Learning | Curtis Hawthorne | TEDxMountainViewHighSchool
- Interview with Tero Parviainen about “Musical Spinners From Latent Space”
- Musical Deep Neural Networks in the Browser by Tero Parviainen
- How to Generate Music with Tensorflow (LIVE)
- Magenta: machine learning in art and music
- Project Magenta (TensorFlow Dev Summit 2018)
- Google Magenta’s “Magenta MIDI” Setup
- School of AI Wizards (Austin McKay, a music AI expert and a data science consultant)
People
- Douglas Eck
- Adam Roberts
- Jesse Engel
- Curtis Hawthorne
- Ian Simon
- Monica Dinculescu
- Tero Parviainen
- Hanoi Hantrakul
Teams
Startups
In case if you found something useful to add to this article or you found a bug in the code or would like to improve some points mentioned, feel free to write it down in the comments. Hope you found something useful here.