Pepper sings!

Pepper is the human shaped robot who is able to move, speak, dance and so on enabled by lots of sensors, and is able to be installed custom developed applications to execute activity just like a smart phone.

Goal of this project:

Final goal of this project was to make Pepper to sing a song. However, at the beginning of the project, Pepper was not open to public so much, so the project was highly confidential. Because of that reason, the project member was only a few. And the developer was only myself. I developed the bridge to connect "VOCALODUCER" developed by Yamaha to Pepper, and developed Pepper's gesture/dance as well as pages in his/her tablet on the chest during the entire process until singing.
"VOCALODUCER" is the backend system to compose song with several parameter and lyrics.

Procedure make Pepper to sing:

Pepper:

1. asks user to input lyrics by using tablet
2. asks user to select melody and music genre from the list
3. then, send request to VOCALODUCER to compose music by using parameters from user
4. receives composed music as a response of VOCALODUCER

Challenges:

The challenges of this projects were following:

• Because at this moment, Pepper was not in the market, so almost no informations about developing applications for Pepper
• Application uses Python to develop, however to me it was the first time to use that language
• Schedule was so tight. Duration was only around 3 weeks to complete it.
• It was my first time to design human shaped robot's gesture.

Bunch of try and error was experienced during this development, however finally I succeeded both compete to develop within tight schedule. The completion was at the event venue about 1 hour before the beginning, though.

Panel Discussion:

• Engadget Fes: 将来のPepperはミスをする？ トークセッション「Pepper新世界、描く未来が違う」

(Released Date: 2014/09/20)

POCKET MIKU (ポケット・ミク)

POCKET MIKU is the first gadget/hardware musical instruments in the world that is able to sing with voice of Miku Hatsune, and the gadget is be able to make sing a song by playing as same way as playing the piano.

My role:

My role in this project was to develop utility application to update lyrics of POCKET MIKU from browser with using Web MIDI API.
At the moment of the project, HTML5 was in early stage, and Web MIDI API was the one of the experimental web browser's API and first API that connect directly to external hardware from web browser. So this utility web application was the first web application in the world targeting for consumer hardware products that controls external hardware with using Web MIDI API.

Movements after shipment:

The most excited thing is that the source code of this utility web application was open to public at GitHub, so using that source code as a reference to develop applications for POCKET MIKU by DIY software developer, such as who loves Maker Faire for example. To develop its applicaiton gradually got be popular, and lots of demo videos for those applications were shared on YouTube.

It was such a remarkable moment for both project members and those who joined this movement.

Challenges:

The challenges of this projects were following:

• To develop with brand new and experimental API to consumer product.
• In the company, it was the first case to open source code to public of consumer products application.

Link to the product page:

• [Product Page]
• [Web application]

(Shipped Date: 2014/04/03)

Web Music Developers JP

Founded this community in Sept. 2013 with awesome 5 co-foundors, @agektmr, @toyoshim, @aike1000, @g200kg, @yukio_tada. The community is grown around 200 members. I am in charge of the organizer of the community.

The word "Web Music", stands for especially mixture of Web Audio API and Web MIDI API, was invention of this community.

Web Music Hackathon:

Most known activity in this community is the "Web Music Hackathon". Here is the reports for each hackathon.

• Japan:
• #1 ウェブはどこまで楽器と融合できるか？ By Eiji Kitamura
• #2 Web Music ハッカソン #2 やりました。
• #3 Event Report: Web Music Hackathon #3 By Eiji Kitamura
• #4 Web Music Hackathon #4 @Kyoto with GDG Kyoto
• #5 Web Music Hackathon #5 @Tokyo
• Worldwide:
• Google Web Audio/MIDI Hackathon at Music China 2015

The hackathon is holding worked with incorporated associations and companies listed below.

• Incorporated Association:
  • AMEI (Association of Musical Electronics Industry: 一般社団法人音楽電子事業協会)
  • MMA (MIDI Association)
  • JSPA (Japan Synthesizer Professional Arts: 一般社団法人 日本シンセサイザープロフェッショナルアーツ)
  • W3C (World Wide Web Consortium)
• Company:
  • Yamaha Corporation
  • Roland Corporation
  • KORG Inc.
  • Google

Meetups:

Meetups with inviting spec editors and others as speaker.

• Immersive Web Meetupを開催したよ！
• Web Music Developer Meetup@札幌レポート

Codelabs:

Here is the step by step tutorial, a.k.a Codelabs, for music related API.
This Codelabs was recognized by Google's Open Source Office.

• Web Audio API
  Almost all of API's demos and step by step tutorial are inside in English, 日本語 and 中文(中国大陆).
• Moco for LUFA (MIDI firmware for Arduino)
  Describing step by step how to make Arduino board into USB-MIDI device.

Panel Discussion and Presentations:

• "Web and Music" (Panel Discussion)
  • Panel discussion at [KEIO SFC OPEN RESERCH FORUM 2014]
    with Jun MURAI, Takuya OIKAWA, Kensaku KOMATSU and Kazuo HIKAWA
  • [W3C Blog]
• HTML5 Conferenceを終えて (Presentation)

Tools:

• webaudio-controls
  Web Components to easily put knob, slider and button on to your web application
• Web MIDI API Wrapper
  JavaScript wrapper to easily use Web MIDI API

HTML5 Experts.jp

As one of HTML5 experts, wrote about 10 articles at html5experts.jp especialy for music related API and Google I/O.

Play Music on Web Browser (Web Audio API, Web MIDI API)

The journal of the Institute of Image Information and Television Engineers. Explaining how to make, to handle and to play music by browser. (Sorry, the journal is not open to public.)

Clock face for Fitbit fitness tracker

Unique features:

• Change clock face from analog to digital on watch screen
• Weather information around you can be displayed on watch screen
• Configuration can be sotred in cloud

See here for more information.

blog post

(Shipped Date: 2018/05/27)

Web Music Browser

This Android app is the web browser supports to send/receive OSC (Open Source Control) message.
OSC is the protocol that are transported across the internet and within local subnets using UDP/IP and Ethernet.

Sending packet form web browser against specific client without any negotiations is still not allowed in 2021. However, would not that great if it is allowed and to develop the controller applications with OSC as web application only with web technology, such as HTML/CSS/JavaScript? Well, this app supports that.

So in short, with this Android app, the applications developed on top of web technology such as HTML/CSS/JavaScript having feature to send/receive OSC message is able to run.

Please refer this document for more API specification in detail.

Google Play and the Google Play logo are trademarks of Google LLC.
(Shipped Date: 2016/08/01)