CCRMA Workshop: 
Designing Physical Interactions for Music

This page contains information about my (Roger's) presentations during the workshop:

1) Musical ideas for various sensor and sound/light output
2) What's inside LinnStrument and how the sensing system works
3) Tips for making an effective project and presentation
4) Making your project into a commercial product



Musical ideas for various sensors and audio/light output


Force-sensing resistors

These are flat pads whose resistance varies in proportion to how hard you press on them. (Here's one on Adafruit.) For example, you might use finger pressure to control a lowpass filter frequency, or use 2 FSRs to control both filter frequency and resonance. You can also derive strike velocity by measuring the peak of the pressure envelope.

To read the pressure of an FSR, connect an analog input on Teensy to both one pin of the FSR and one end of a 10k resistor. Then connect the other pin of the FSR to the Teensy's 3.3v pin, and the other end of the 10k resistor to the Teeny's Ground pin. With no pressure on the FSR, the analog input will read 0 volts. With full pressure it will read about 3 volts.

Flex Sensor

These are long strips that change resistance between on how much you bend them. (Here's one on Adafruit.) For example, you could tape flex sensors to measure the bend of your elbows, knees and neck, in order to shape synthesized sounds based on arm and leg movements. These could also be used as a pick detector to sense the action and strength of picking as on a guitar string.


Accelerometer

These devices sense movements in three dimensions: pitch, roll and yaw. The easiest way to implement an accelerometer into your project is with the Prop Shield described above. For example, you could hold the Prop Shield in your hand and use Roll to vary oscillator frequency, Yah to control filter frequency, and Pitch to control filter resonance.
Links:
Product page for Prop Shield: https://www.pjrc.com/store/prop_shield.html 
Video demo: https://youtu.be/5mccPaAEya8?t=24 
Pitch modution: https://youtu.be/cTBMhoi8nhc?t=118


Force-sensing linear potentiometers

These are linear strips that sense both finger position and pressure when you touch the strip. For example, one of these could used to create a musical pitch strip that produces a sound when pressed, with pitch determined by position and loudness determined by pressure. Four of these strips could be made into a 4-string musical instrument. Here's a 4-inch example for $10. These sensors actually sense either pressure or position at a time, so your code needs to rapidly switch between sensing each in order to appear to sense both.


Force-sensing trackpads

These are trackpads that also sense finger pressure. For example, one of these could used to create a musical surface that produces a sound when pressed, with pitch determined by X-axis, filter frequency by Y-axis, and loudness determined by pressure. Here's an example for $18. These sensors actually sense either pressure, X-axis or Y-axis at a time, so your code needs to rapidly switch between sensing each axis in order to appear to sense all simultaneously. Links:
https://www.tangio.ca/single-touch-trackpad-force-sensors 
3D drum strike control: https://youtu.be/HHWrSs8o1V4


Force-sensing multi-touch trackpads

These sense the position and pressure of multiple touches, allow the sensing of chords. https://www.tangio.ca/multi-touch-neo-sensing  


Joysticks

Similar to a trackpad, you can use a 2D joystick to simultaneously control 2 synth parameters with one finger:
https://www.adafruit.com/product/2765 


Distance sensors
‍‍
These are sensors that measure the distance from the sensor to an object like your hand. For example, two of these could be used to make a Theremin, with one measuring left hand distance for volume and the other measure right hand distance for pitch. Here's such a sensor on Adafruit.


Light and color sensors

These are sensors that sensor the brightness of light, as well as the brightness of red, green and blue of that light source. For example, you could alter a synthesizer sound by altering the color of light shining on it. Here's such a sensor on Adafruit.


Air pressure sensors

These air pressure sensors can be used to create an electronic wind instrument, with your breath pressure controlling sound volume. Here's such a sensor on Adafruit.


Depth Camera

Sense 3D shapes like hands/fingers, objects:
https://www.sparkfun.com/products/17770 



Musical ideas for sound/light output

Neopixel strips

These are long strips of addressable LEDs, allowing you to individually control the color and brightness  of each LED within the strip. These can produce engaging light animations.
Here's one on AdaFruit: https://youtu.be/gJzwjhVEqBE
And here's an example of using a Force-Sensing Linear Potentiometer (see above) to move light along a Neopixel strip: https://youtu.be/gJzwjhVEqBE
See above on this page for info on asssembled Teensy and Prop Shield, and assembling the parts for a fully-working LED strip controlled by a Teent Prop Shield.


Teensy Audio Library

The Teensy 4.0 is fast enough to process audio in real time. When used with PJRC's Audio Shield, the Teensy becomes a self-contained stereo audio processing device. Here's the Audio Shield:
https://www.pjrc.com/store/teensy3_audio.html 
PJRC has also created a wonderful software tool called the Audio Library:
https://www.pjrc.com/teensy/td_libs_Audio.html 
The Audio Library is a visual programming interface like Max/MSP or PD. You can simply drag audio modules from the large library of synththesizer or audio effects, connect them with visual wires, then push a button to generate the code that you copy into your Teensy editor. Viola! Instant synthesizer or guitar effects box.


What's inside LinnStrument and how the sensor works

I make an expressive music keyboard called "LinnStrument":

https://www.rogerlinndesign.com/linnstrument

And here's a video of what musicians are doing with it

https://youtu.be/MDTikW1BFt8

In this talk, I'll discuss why I made it, how I made it, using a 3D modeler, what's inside, how the touch sensing system works, and how I designed to be manufacturable.

Here are some pages I'll reference in the talk:

Design goals and choices

How the sensor works

Early Design concepts



Tips for making an effective project and presentation

My planned talk for Wednesday was "Creating a music product, part 1: How to create a proof of concept of your product idea at low cost". However, I've decided it would be more helpful to give some suggestions on how to make your projects most effective and to give an effective presentation, including time for Q&A about these topics. And I'll include my thoughts for "How to create a proof of concept of your product idea at low cost" in my Thursday presentation.

Here are some tips on making an effective project and presentation:

* If you haven't yet selected a project, here are some sites with ideas:
1) A page with photos of projects from the 2023 workshop: https://www.icloud.com/sharedalbum/#B16GRMtznGOT6Va
2) The Guthman Competition, an annual competition for instrument designers at Georgia Tech: https://guthman.gatech.edu. These projects are more elaborate than your can create in our short workshop, but might spark an idea.

* Simple ideas are best. The best instruments are simple to use and understand, but allow development of skills through practice.

* Get it functioning first, then make it pretty. In the short time we have, it's important to solve the technical problems first. Don't waste time at first on making it pretty, or you might be scrambling at the last minute trying to get it working.

* Make it reliable. Murphy's Law: your project will alway fail just before you do your presentation. Use rigid wires instead of flexible wires so that they won't accidentally get pulled out. Any cables to the project board will get pulled, so put a nail in the board and wrap the cables around it, so that cable pulls won't rip all the small wires out of your breadboard. Practice moving the entire project then turning it on, to see if it still works each time.

* Stop, breathe and think to solve problems. Often people will spend hours trying to make their code work, only to later discover that the problem was due to a resistor lead not being pressed fully into the hole in the breadboard. Test your connections with a beeping muli-meter.

* Practice your presentation. Clearly and concisely state 1) what it is, 2) why it's useful, and 3) tech details of how you created it, then 4) give a short and effective performance that shows the merits of the idea. Time it to be certain you won't run over. Practice your presentation on a friend first, then ask if he clearly understood it. Practice is important so that speaking in front of the class doesn't steal your brain function. :)

* Sasha and I are here to help. If you're stumped, we are likely to know how to solve the problem.

* Q & A? 


Making your project into a commercial product


After you make your project, you may be thinking "My project idea is very cool. I think other people would like it. I think I'll make it into a commercial product. If so, here are some tips:

Start small. Don't quit your day job. Take it a step at a time. Don't worry about investment, patents, business licenses, regulations, expensive industrial designs, etc. These things will simply slow you down. Start the product as a side hobby, working evenings and weekends, and focus on learning if other people value your idea as much as you do.

First, streamline your design a little so that it's easier to make in small quantities. For example, mount your Teensy board and controls in a stock box with holes drilled in it for knobs, etc. Make sure it survives drops and being throwing around in a shipping truck. Then try it out on friends to see if they find your UI useable, and guess at a price that is attractive enough for people to buy it. Then build 10, make a web site, send an announcement to some music product web sites, and see how many people buy it. If you sell out, build 25. If those sell out, build 50, etc. Use the money from each build to fund the next build.

Each time you build more, you'll have learned from your customers about what changes are necessary. Refine your design with those changes. With each new build, you'll get closer to what the correct design and price is. For this reason, it's best at first to not to commit to any design elements that are expensive to change.

If you keep making more and people are still buying more, then it's time to make it more manufacturable. Make a circuit board and design that is simple and low-cost to manufacture and test at lower cost.

Even at this stage, I still wouldn't worry about patents. First, it's unlikely that your idea is patentable. Also, patents take a lot of time and money and even if you get one, all it does is give your a better legal case when you must hire a lawyer to sue the infringer. Also, at this early stage I wouldn't worry about business licenses or regulations. No agency will care until you start making money. Plus, electrical/emissions/environmental regulation testing and certification is expensive, and every time your change your design you'll have to re-certify it.

Avoid taking investment. If you do, your company mission statement changes from "I want to make cool things" to "I need to make products that will pay my investor a multiple of his investment".

Try to use low-cost off-the-shelf parts wherever possible:

* Cheap metal boxes from Hammond Manufacturing.

* Try to use parts that are commonly available from digikey.com, mouser.com, McMaster.com. If you choose a rare part, you may find that your entire production is held up for a single part that is out of stock.

* For front panels, either use silk-screen printing or for a slicker look, use pre-printed adhesive labels with pre-cut holes for knobs and panel controls.

* Avoid custom processes with high up-front costs like injection molding. If you need to change your design, you'll have to pay those up-front costs again. Instead, use stock boxes or sheet metal, wood, or 3D printing.

Also, try to design the product for easy owner-servicing by following a video. If a customer problem arises, you can simply send him a replacement circuit board or other part that he can replace himself, which is much cheaper and more convenient than having to return the product to you for repair or send it to a repair shop.

On your site, make lots of videos about how to use it.

Instead of a printed manual, put your entire manual on your site, including videos, FAQs, etc., and keep them updated.

Build into your product an easy way to update the software. There will be bugs.

Most importantly, have fun!