has joined #papers02-bespoke-adaptable-admis

@joe.wright set the channel purpose: Paper Session 2: Bespoke / Adaptable ADMIs

has joined #papers02-bespoke-adaptable-admis

has joined #papers02-bespoke-adaptable-admis

has joined #papers02-bespoke-adaptable-admis

has joined #papers02-bespoke-adaptable-admis

has joined #papers02-bespoke-adaptable-admis

has joined #papers02-bespoke-adaptable-admis

@joe.wright has renamed the channel from ?papers2-bespoke-adaptable-admis? to ?papers02-bespoke-adaptable-admis?

has joined #papers02-bespoke-adaptable-admis

has joined #papers02-bespoke-adaptable-admis

has joined #papers02-bespoke-adaptable-admis

has joined #papers02-bespoke-adaptable-admis

has joined #papers02-bespoke-adaptable-admis

has joined #papers02-bespoke-adaptable-admis

has joined #papers02-bespoke-adaptable-admis

has joined #papers02-bespoke-adaptable-admis

has joined #papers02-bespoke-adaptable-admis

has joined #papers02-bespoke-adaptable-admis

has joined #papers02-bespoke-adaptable-admis

has joined #papers02-bespoke-adaptable-admis

has joined #papers02-bespoke-adaptable-admis

*Rough-hewn Hertzian Multimedia Instruments* _Marinos Koutsomichalis_ Paper 122 in proceedings folder *A Dimension Space for the Evaluation of Accessible Digital Musical Instruments* _Nicola Davanzo, Federico Avanzini_ Paper 41 in proceedings folder *A Survey on the Use of 2D Touch Interfaces for Musical Expression* _Diemo Schwarz, Abby Wanyu Liu, Frederic Bevilacqua_ Paper 38 in proceedings folder  *The Longevity of Bespoke, Accessible Music Technology: A Case for Community* _Alex Lucas, Miguel Ortiz; Franziska Schroeder_ Paper 46 in proceedings *Cyclops: Designing an eye-controlled instrument for accessibility and flexible use* _William C Payne, Ann Paradiso, Shaun Kane_ Paper 112 in proceedings

An updated version of paper 38 *A Survey on the Use of 2D Touch Interfaces for Musical Expression* is here: https://hal.archives-ouvertes.fr/hal-02557522v2

Supplemental material (the survey itself): https://hal.archives-ouvertes.fr/hal-02557522v1/file/2D%20Touch%20Input%20for%20Musical%20Expression%20Survey.pdf

** We will start with paper session 2, webinar 2, in 3 mins, see you there!

Please ask questions for the authors on here, @joe.wright will be keeping an eye out on here, and will relay questions for the Q&A

Zoom asks me for a meeting password?

Thanks @niccolo.granieri :smiley:

It shouldn't, probably a zoom glitch because the room is not open yet.

I think it?s the last part of the link *pwd*

clicking on the link should suffice :slightly_smiling_face:

clicked again, now seems to work.

thanks

Happy to see it was a glitch :slightly_smiling_face:

it asked me for my email address, not a password, before getting me to the holding area before webinar starts

Yes @a.macdonald, that happens when you're not logged in with a zoom account on your default browser. It asks for email and display name to give an idea to the organisers of who's joining.

we're live :smile:

Nice:+1:

Here is the link for live captions:

https://www.streamtext.net/player?event=NIME220720

I think you may need to mute Julia: i can see this at the top of my screen :slightly_smiling_face:

Thanks Corey, on it!

that was supposed to be ?within which? not ?within weeks? :slightly_smiling_face:

Noted!

Lol, sorry! Muted now

Thanks Julia

?ancient Messene? not ?machining? too :slightly_smiling_face:

The captions in the videos are not me, I?m only doing the live non-video bits :)

ditto, sorry for this Marinos. Captions will be corrected after the panel.

thanks for bearing with us! Dont forget to post questions for our paper authors :slightly_smiling_face:

no worries, my pronunciation doesn?t probably help.. there are a few other things too - we can correct them afterwards together !

That would be awesome! Thanks so much for understanding. If you DM me after the panel we can go through it together :slightly_smiling_face:

Thanks for the presentation, Marinos! The question is focused on the first instrument you presented - collecting different material, images, text, sound, etc, is there any attempt to create a synesthetic coherence between the different media, or do you treat them as individual expressions?

I?m unfamiliar with the term ?hertzian?. Can you describe this perspective?

Interesting paper! it makes clear the aesthetic starting points that go into creating new instruments, which is quite important, but not always made explicit in NIME papers. Whether you come from an instrumental/performer perspective, a composer/composition perspective, or critical media art perspective makes an important difference in the design choices that are made along the way! In that sense, the question where are you coming from? is as important as what do you want to do? And this is also important in how to evaluate or view NIMEs

Thanks!

thanks, that's great, will read the paper!

Once you're here @vincze, click on the above link to join :slightly_smiling_face:

Very interesting work, thanks @m.koutsomichalis

better send me an email as I have to rush to an urgent meeting just after the panel.. sorry for this.. or

love the experimental brutalist look of this project!

Thank you! @niccolo.granieri

thank you @info041 - that?s quite the proper aesthetics here

thanks @mark.durham

@marije I certainly agree with that - from point of view and what is particular interesting for me is how instruments can be the means to critically reflect on the ?canonical?standard ways to perform, compose and design instruments as well as - most importantly - to explore non-standard and experimental ways to do so. In this sense, where one?s coming from is a very important point of reference that comes with its own set of strategies for experimentation and contextualisation.

thanks @manolimoriaty:)

Keep your questions coming if you have them!

This one is scheduled until when exactly?

For the physical impairment: how would instruments that address multiple levels? In a project I worked on at STEIM, we made an instrument that could be used at different levels, using shape to allow for different affordances, in combination with an adaptable sensitivity.

Hi @vincze , everything is available on the timetable. This specific session finishes at 13:45 UTC+1

Thank you @nicola.davanzo . It is great to see you thinking about how to provide a framework to help NIME researchers/builders make sense of and work in this space. My question is whether you consulted people who play ADMIs at all in this work? I imagine they would have some great insights into what they perceive as relevant dimensions...

Great presentation, ty.

Thanks for this interesting talk. To what extent do you think the location on the dimension space is an inherent property of the instrument, versus something that might depend on the context? The same instrument can be used by different musicians in different situations.

Very useful characterisation! I wonder if there is first data/experience about which design choices best improve the well-being of the users.

this is the STEIM project: https://vimeo.com/170797291

Were users with access issues involved in the process of creating this work?

I'm not sure whether the highest level of impairment is a good measure though... as the level of lowest impairment has quite different needs, especially when combined with cognitive impairments. Whether an instrument breaks when you treat it uncarefully then becomes quite important.

There is a danger in the field of design for access where users can feel designed 'at'. Prescriptive design without consent.

Thanks @nicola.davanzo for the presentation. One of the uses of design spaces for characterizing (A)DMIs is to identify "under-explored" areas. Did your dimension space highlight any areas/locations that were not represented in the literature?

Joseph: definitely instruments dedicated to sensory impairment. There are few works which insist on this "axis". In our paper we included an example of one dedicated to hearing impaired people, which involves interaction with images: _M. S. Grierson. Making music with images: interactive audiovisual performance systems for the deaf._

Also, few instruments are really adaptable. Clarion is a good example of an adaptable instruments, but few ADMIs are really adaptable to different users. One great advantage of an adaptable instrument, over using different kind of instruments with the same group of people, is the social component. Clarion shares the same interface for all its users, making it interesting for group experiences

Love how blues, country and jazz are the "exotic" genres with this crowd :D

Any questions for Diemo?

looking at the devices mentioned, most of them have a third dimension as well, any particular reason why that hasn't been taken in consideration?

Feed is breaking up for me, anyone else has the same?

Not here Manoli

and also, can you share a bit more info about who took part to the survey?

Sorry Manoli, all good here!

me neither, is it audio and video?

Not for me!

no problem, it might be my internet, has been odd lately

Thank you for this warning! As a too young researcher, I didn't know about this.

My experience has been that in many musical contexts, musicians have difficulty thinking of what hypothetical instruments they might want, other than variations on what they already play. @a.martelloni found something similar with percussive fingerstyle guitarists and I've found it with violinists. Do you as a designer have a personal perspective on what new 2D(+) instruments might be created based on these survey results?

In case you missed the surve, are using 2D touch devices, and want to correct the population biases in the study, it is still open for your insights: https://docs.google.com/forms/d/e/1FAIpQLSelcPlfYC5ngImjMC0S8QV1-Ry5RcWeuGUZfsIi-EQN95jidQ/viewform

Did you examine the number of fingers that were typically used at one time with touch devices and how this may have been considered when interacting with the device?

might the popularity of generic controllers with 2D interfaces over music-controlllers with 2D interfaces possibly have to do with the MIDI resolution of the latter?

Thank you for the question! Actually not, for this work, and I agree that's a pity. I mainly researched in literature to understand which were the main design features involved. Actually, here in Italy I had some difficulty in getting in contact with such people. I had contact for example with a man having a late stage SLA, which communicates only with an eyetracker, but actually he contacted me because he didn't know about gaze controlled DMIs... I felt a great gap between people/rehab centers and technology

Thanks @schwarz. Really interesting research.

thanks, great presentation!

@nicola.davanzo you might look into "Nothing About Us Without Us" (https://en.wikipedia.org/wiki/Nothing_About_Us_Without_Us), as a slogan that suggests certain approaches in work around disability/access. I think it would be great to explore where such an approach--namely engagement with people who use ADMIs or who might be interested in using ADMIs--may lead your research next!

good question, @marije, but I think there is not a lot of MIDI use in the participants. Most specialised music controllers already send good resolution data over USB.

I'm guessing quite a few of these still use MIDI-over-USB.

On the point of people wanting larger 2D surfaces. When @v.zappi and I built a super-reductive instrument with a small touch sensor and a deliberately narrow pitch range, one piece of feedback from players was that they wanted a longer touch sensor "so they could play more notes" (or something like that). It's funny that even though people know the instrument is programmable, they still intuitively think that a larger surface is needed to access a greater control space.

Another issue might be the longevity of these music controllers (or lack of them).

We?ve had some great questions so far in this session, keep them coming!

Thanks @schwarz. In my experience, the advantages of 2d touchscreen interfaces (tablets/smartphones), such as bespoke interface creation and customisation, are still far outweighed by the lack of tactility, resulting in the requirement to always be looking at what you are doing with your hands. This is particularly problematic when trying to create interfaces for MIDI note generation (a basic example being a digital touchscreen representation of a traditional piano keyboard), but also has implications when thinking about buttons, switches, and pot/fader banks etc. I was wondering what your thoughts are on this and whether your foresee any way to overcome these limitations in the future. The ROTOR controllers made by Reactable (https://reactable.com/product/rotor-controller-set/) are one example I have seen, but the fact that they sit atop the screen without being affixed in any way still means that interacting with the control knobs using only your ears and without looking at what you are doing is not really viable.

Really great presentation. Asking the right questions! Assessing the right needs. Longevity is key and a tricky beast!

that's definitely true! Luckily, some control paradigms are stabilising (2D + pressure)

will you keep working on this instruments? or is the main idea that of reusing code and/or components/tech?

I wonder if this is a larger HCI problem along the lines of "optimal size for a tabletop or lap-held touch surface"? Leaving phones aside (as they are meant to fit in a pocket more than be 100% comfortable to use)... How large starts being too large? I think Lia's study is again somewhat relevant, but I would look more specifically into why things like the Launchpad (or tablets for that matter) are the size they are!

Great presentation @alucas02 and great to see Eoin again. I was in a DnB band with his brother... :wink: In some of the work I've done with the Open Youth Orchestra of Ireland we took the approach that new music technology belongs within an ecosystem of performance, composition and participation. Do you think that the a breakdown of that ecosystem could hinder the longevity of ADMIs? Or perhaps DMIs in general?

Important aspect: about setting up in the home context, and developing in that context. The caregivers are usually not trained to give the kind of support, or used to reporting back about issues that show up. This makes the development of these types of instruments a big challenge, as the move from a prototype instrument to an instrument that can be used on a long term basis, is a lot harder, given the possibilitilies of feedback from the users (both the musician and the caregiving context)

Searching through literature, I saw that there's an amount of request for augmentations of acoustic instruments: actually our design space axis "design novelty" does in fact not indicate that an instrument designed from scratch, with a completely different design than the acoustic one would be better than an augmentation of the acoustic version (the opposite is also not true!). Mine is just a curiosity: was Eoin specifically looking to be able to play guitar? Would he feel better to have a real augmented guitar with actuators or it was indifferent for him?

On that topic, did the research distinguish in any way between instruments like Sensel Morph or Joue that have "variable tactility," as opposed to screens like the iPad, which are all about visual feedback. Or, Wacom tablets, which have neither.

in the previously mentioned STEIM project, that caregiving context consisted of the music therapist, the nurses, and the family of the musician.

great remark, @meikleg! some controllers have eyes-free tactility (Joué, Sensel with overlays or DIY tactile guides), some allow relative positioning by their surface texture (Linnstrument, Lightpad M).

@alucas02 Thanks for presenting, so beautifully, the work of "Performance without Barriers".

@schwarz Very interesting! It's encouraging that the acoustic excitation model is still largely overlooked :slightly_smiling_face: . From our point of view, the texture of the surface is key.

@meikleg This is a really good point. I almost exclusively use touchscreen interfaces (Android tablets) for performances, because for me the ability to rapidly create a high resolution bespoke interface that can be physically decoupled (OSC over WiFi) from the computer screen is important. However, the tactile nature of "traditional" instruments and other controllers helps to navigate around the interface when perhaps visibility of a touchscreen interface is obfuscated, and this is something that I definitely miss not having access to.

@alucas02 Thanks for your paper! I wondered if there are any tools or existing organisations (Like Drake or Instrument - Maker) that you wanted to flag up in the chat that people can look to as starting points / exemplars for developing more sustainable tools/communities around ADMIs?

Thanks for your response @schwarz. I'm aware of the Joué and Sensel Morph, while I have a Lightpad Block. I guess the problem with the first two examples is that the creation of overlays is less accessible to most users than using something like Liine Lemur or TouchOSC on a tablet to create a custom interface, while the Lightpad Block still suffers from the same problems as a tablet screen, in that it isn't possible to differentiate between individual parameter controls on the interface by touch alone. I wonder if there is scope for something like haptic feedback to be used to provide individual parameter controls within a touchscreen-based interface with a unique sensory signature, making orientation within the interface without the need to look at where your hands are a possibility?

I know you mentioned some in your paper too!

any plans to make Cyclope interact with other instruments (ie via MIDI)?

@william c payne - have you experimented in ways to promote expression? (volume, timbre, envelope etc..)

really nice demo

Really happy to see that gaze based instruments are still researched! They still have a lot of unexplored potential imho, and its nice to see you explored other features than note level control. About the last one, did you experiment with different keyboard layouts?

I'll suggest the Vodhran (see demos tomorrow) for an example of a 2D interface (Sensel Morph with contact mic.) being virtually perfect for a specific instrument design. We're aligning with a hand percussion instrument and use the Sensel to supplement info we can't capture from the contact mic. alone.

So much of it is about what your design goals are...

Does eye-blinking have a role in eye-gazing technologies? In the project MotionComposer, eyeblinks are used a way to control music playing.

Hiroshi Ishii to the rescue :smiley:

The "radical atoms" concept applied to a touch controller would be THE thing.

thanks!

Thanks @schwarz, I wasn't aware of this. Really cool stuff!

@william.payne very interesting work

Thanks everyone, a really interesting session!

Hi @william.payne! I'm also a William Payne working in accessible music tech. How weird!

Hi @mario.buoninfante, there is more detail about the target groups and the actual turn out in the paper. It surely is not representative of the community =-(

UU/RIAM ran a project last year to create the Open Youth Orchestra of Ireland. The project was called Le Cheile (together in Irish). It created three new inclusive ensembles around Ireland (with UU already having one). The esnembles are individually run by each city's institute and come together to form OYOI. The project coordinator was Brendan Breslin at RIAM. He's always happy to share insights into that project. It continues with 5 year funding. I'm not sure if RIAM have put any details on their website but it may be worth checking.

Very interesting session, thanks everyone !

Disclosure... I was tech lead on that project.

Oh my gosh - that is so amazing! We should start an annual William Payne gathering or something

See also @abi?s research from ICMC 2017 on how different size DMIs change the way music is played on them.

Thanks @schwarz for an interesting presentation!

Thanks for your great example to follow!

It seems there were some similar results to our own - I look forward to reading the full paper now. Also, +1 to your comment along the lines of ?you always find the right questions to ask after you get the results back?

@schwarz thanks for your talk! The lack of enthusiasm from respondents for machine learning was surprising, given how trendy it is. Did you get any impression from the data that this might be affected by varied connotations of what ML implies? I.e. people might be associating with big, cumbersome or opaque modelling processes, rather than (say) lighter-weight things like dimensionality reduction.

Hi @schwarz, just saw the message. I'll check out the paper, thanks :slightly_smiling_face:

great work btw

@william.payne, really enjoyed your presentation. I notice in your videos that you've also used Cyclops to control acoustic instruments and wondered how successful that was? It'd be great to have more instruments that get out of the digital domain.

also, for time reasons I couldn't mention your preceding work properly in the presentation, just give a shout at the end, but it is acknowledged in the paper, of course.

certainly a lack of knowledge about ML leads to prejudices. There were mentions of non-deep ML like knn, NMF.

We do have the possibility to ask for more details, most participants consented to follow-up questions.

Hi all, thanks for the remarks and fruitful questions to the 2D survey article. We do have the possibility to ask for more details, as most participants consented to follow-up questions. I occurs to me that here would be a good forum to collect questions to ask them. So far I gathered: ? When fingers are used, is it one finger or multiple? (@james.dooley) ? more detailed reasons why ML is not used (@o.green) ? something about if limitations of MIDI play into the usage (@marije)? Feel free to brainstorm as thread replies.

Thanks, both. Here are some links to communities of makers working on accessible technology more generally: http://atmakers.org/ http://enablingthefuture.org/ https://www.makersmakingchange.com/ https://www.remap.org.uk/

Specifically relating to inclusive music, the organisations below are involved in some exciting work: https://www.ohmi.org.uk/ https://www.drakemusic.org/technology/ http://sharemusic.se/english/ https://www.heartnsoul.co.uk/ https://www.humaninstruments.co.uk/

The following links are to tools for inclusive music which are etiher in active development or have seen sustained use. Instrument Maker: https://github.com/matthewscharles/instrument-maker Kelly Caster: https://www.drakemusic.org/technology/instruments-projects/the-kellycaster/ Adaptive Use Musical Instrument: https://paulineoliveros.us/activities.html Sound Control: http://soundcontrolsoftware.com/ Eye Harp: http://theeyeharp.org/

Can't believe I forgot ShareMusic

Don't think I know all those Alex. Good share.

Thanks both, if this keeps growing I might copy it to a channel!

See also @Shelly Knotts talk "A survey on the uptake of Music AI Software" tomorrow.

of course, and thanks for the acknowledgement. I?m happy that the work could contribute beyond its own immediate aims.

Hi Tim - thank you for the question! The Cyclops is computer based only, but we had talked about extending it with actuators and additional LEDs to add presence. To your point, controlling physical instruments can be very effective. Shaun Kane, my co-author and mentor was behind an eye-gaze drumset. It was thrilling for Jeremy to use it. As someone who can?t physically manipulate things in the world and must rely on a computer for speech, he was able to make a loud statement with the drumset.

The Hands-Free Music project has been working on instruments for a while now: https://www.microsoft.com/en-us/research/project/microsoft-hands-free-music/

Hi Willie - ah that makes sense! I looked at some of your videos and assumed it was the same instrument.

The research project looks amazing. I'd love to know a bit more. Maybe we could catch up on Zoom at some point? I'm presenting the keynote this evening with Drake Music talking about our ADMI work - hopefully it'll be of interest :slightly_smiling_face: I'm sure there's scope for collaboration ...

And working with children and young people, tools have to be absolutely bulletproof before they are usable for schools, as support staff come and go. It's frustrating to develop a great tool and then your student cannot use it because a relief worker turns up and is overwhelmed by the setup instructions. And that last stage of turning a prototype into a a proper one-click executable is the stage you can never get funding for!

@william.payne I have a student who I think would really enjoy Cyclops, is it available somewhere? She uses Tobii on a Microsoft Surface and is familiar with similar instruments, this would give her something musically new.

@alucas02 really enjoyed your presentation! I am from Drake Music Scotland, and have been doing some development of eye gaze interfaces, but I don't have the coding background to expand on the design. Would love the idea of creating a central code bank (probably not with my ugly code in it!!). Would love to work with you more. And the family would be thrilled to see NI again.

Agreed, it is usually where the frustration kicks in for eye gaze users when they find the music cannot be as expressive as they intend. Really need options to grow and fade a note, perhaps with dwell, intensity of fixation or standard x,y positioning. Open to hearing about anything that can offer expressive capability to eye gaze players.

Thanks Clare, I'd really like to chat with you and DM Scotland. You'd be very welcome to visit when the lockdown eases. I also sometimes find myself in Scotland as my inlaws live in Aberdeen!

@x Excellent point! I feel like my disability is connected to my drive to build accessible instruments and we need to find ways to empower more makers

@nicola.davanzo Its a learning curve for all of us - no matter the field. I recommend connecting with Amble Skluse (https://www.ambleskuse.net/) and her paper.