Video
Description
스마트하게 음악을 눈으로 즐기 수 잇는 프로젝트입니다! Google Speech api를 이용하여 SoundCloud의 음악을 검색 한 후, Play하면 사운드를 LED Matrix에 시각화 해줍니다. 우리는 이 프로젝트를 위해 소프트웨어 측면에서는 Google Speech api, Python, Gstreamer, Cava를 이용 했고, 하드웨어는 라즈베리파이와 32 X 32 LED Matrix를 이용 했어요! 함께 음악을 눈으로 즐겨 볼까요?
This project is about visualizing sound pulses on RGB LED Matrix. We use Google Speech API to convert speech to text, use it as query parameter for SoundCloud Api, and finally playback the recommended song using Gstreamer player, and at the same time show visualization on RGB LED Matrix.
Until now, we have achieved first milestone of simulating the visualization using Cava on raspberry pi's terminal. Showing actual output on RGB Led Matrix (32x32) is work in progress.
Libraries and Hardware Used
- Raspberry Pi 2
- Google Speech API (Python library)
- Cava
- Gstreamer player
- RGB LED Matrix (32x32)
- Bluemoon USB Mic
- Speakers
How to run this code
Setup
Before running these samples perform the steps:
-
Clone this repo
git clone https://github.com/DjangoGirlsSeoul/BeatBeat.git cd BeatBeat
-
Create a [virtualenv][virtualenv]
virtualenv env source env/bin/activate
Install the dependencies
pip install -r requirements.txt
Google Cloud Speech API
Prerequisites
Enable the Speech API
If you have not already done so, enable the Google Cloud Speech API for your project. You must be whitelisted to do this.
Set Up to Authenticate With Your Project's Credentials
The code uses a service account for OAuth2 authentication. So next, set up to authenticate with the Speech API using your project's service account credentials.
Visit the Cloud Console, and navigate to:
API Manager > Credentials > Create credentials >
Service account key > New service account
.
Create a new service account, and download the json credentials file.
Then, set
the GOOGLE_APPLICATION_CREDENTIALS
environment variable to point to your
downloaded service account credentials before running this example:
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/your/credentials-key.json
If you do not do this, the REST api will return a 403. The streaming sample will just sort of hang silently.
See the Cloud Platform Auth Guide for more information.
SoundCloud
Obtain API Key (client id) from here[https://developers.soundcloud.com/] and add to speech_streaming.py
Troubleshooting
PortAudio on OS X
If you see the error
fatal error: 'portaudio.h' file not found
Try adding the following to your ~/.pydistutils.cfg file, substituting in your appropriate brew Cellar directory:
include_dirs=/usr/local/Cellar/portaudio/19.20140130/include/
library_dirs=/usr/local/YourUsername/homebrew/Cellar/portaudio/19.20140130/lib/
Run the Code
-
To run the
speech_streaming.py
code:$ python speech_streaming.py
You should see a response with the transcription result and Sound cloud track result.
The code will run in a continuous loop, printing the data and metadata it receives from the Speech API, which includes alternative transcriptions. Say "stop" to exit the loop.
Note that the
speech_streaming.py
sample does not yet support python 3, as the upstreamgrpcio
library's support is not yet complete. To see the audio visualization simulation.
git clone
the cava respository and run it in a new terminal.
Deactivate virtualenv
deactivate
TO-DO
- Instead of showing output throug Cava, our goal is to show sound visualization on RGB LED Matrix
LICENCE
MIT