Tag Archive for webcam

fluid blobs

August 8, 2009 Projects 2 comments

Linked below are some early results from a new series of sketches I’ve been working on using Processing.  These sketches continue in a long line of projects I’ve completed recently using simple camera tracking algorithms to infer interesting patterns of movement in urban spaces.The first example is a calibrated blob tracking experiment, using the excellent and very well documented OpenCV library for processing.  A few simple modifications to the setup parameters allow for a very customisable tool, able to withstand many of the constraints live webcam installs can throw up.  I’ve tested this in a number of places (my bedroom wall, lit by a single lamp tends to be the best contrast) and will have more to say on the nature of live webcam video in the future.

OpenCV blob tracking – calibrated from Jason McDermott on Vimeo.

The second example is a first attempt at combining the live blob tracking with the wonderfully funky and playful MSA Fluid library also for processing.  This lib is geared towards touch screen interfaces and screen based mouse interactivity – but I immediately thought it would be the perfect partner for my webcam based projects (or even accelerometer/phidget/slider/midi sensor data).  It wasn’t very difficult to swap out the mousex/pmousex variables for centroid x/y data, so the first test has been deemed a success.  I showed this yesterday to Frank/Ale/Amy/george/anyone who would stop for more than 2 minutes in the interactivation studio and it was a big hit :)

OpenCV + MSA Fluid (Processing) from Jason McDermott on Vimeo.

The third example is significant for a couple of reasons – it is another combination this time using recorded video of an actual installation space (filtration fields / DAB courtyard) thus requiring another version of the calibration – but also my first experiments in putting together an arrayed interface between the blobs and the fluid.To explain further; Firstly it’s easy to switch out the mouse for ‘something else’ and inferring movement velocity for a single object/blob is simple.  Secondly I wasn’t so sure about the way to apply this singular blob mousex/pmousex-esque technique to many objects at once.  Thirdly I wasn’t sure if it would all explode in one big fluorescent, particle mess!

OpenCV + MSA Fluid (processing) test 3 from Jason McDermott on Vimeo.

[Update]< <note, v3 will be embedded when vimeo uploads my video. since when does a new video have to wait in a queue for 30 minutes??>>In the end I’d say it’s mission accomplished, certainly with calibration tweaks to occur before I’m happy to unleash this on an unsuspecting public.  I’d be interested to see how this could influence peoples’ behaviour in the space – whether or not we would see people dancing/swimming/painting the space of the courtyard.  I’m curious also to see how this kind of new interaction with the space of the DAB could filter into a new perception of the building as not merely a space to move through but one which is open to new forms of physical conversation.

Related Posts:

Janus

August 6, 2009 Projects One comment

As part of the Smart Light Sydney Festival, May 2009, Tom Barker (Professor of Design, Architecture and Innovation at UTS) and Hank Haeusler (Post-Doctoral Researcher at UTS) were commissioned to design and produce an interactive light sculpture to be exhibited on the light walk in the Rocks.  The piece conceived by Tom was called Janus and was pitched to the SLSF body as;

a giant floating human face in The Rocks..inspired by Janus, the Roman god with two faces, Barker and Haeusler’s installation is part of their ongoing research into complex and non-standard media facades.  Janus uses social media and new technologies to engage the public and influence its art. Photovoltaic cells are used to power the installation.

IMG_1095

The concept for the project was for the face sculpture to act as a mirror to the emotions of the city, as measured using the social media of mms, email and blog updates.  Toms’ earlier research had lead him to explore notions of the nature of facial expressions, our abilities to read and emote via the expressive capabilities of our faces.  With this in mind, it was an interesting experiment – is it possible to measure, collect and respond to accumulated faces – can you determine how happy a city is by watching its’ inhabitants facial expressions?I was invited to join the project as the software design component of the project, as Tom had seen some snippets of my interaction design work, as well as the work of my students in the computational environments class.  Naturally my first thought was to ask Frank Maguire if he was interested in joining me on the project – having worked with Frank on the Filtration Fields installation, his industrial design skills and generally snappy logical mind made him the perfect partner in crime..

The main crux of the project production from our end was in coding the algorithms which would translate images of faces into emotional readings (happy, sad, surprised, angry, fearful, disgusted and neutral), using these readings to trigger pre-recorded videos and controlling the video output to a non-rectilinear array of 192 pixels.Having worked frequently with camera images, facial emotions I was confident in that component of the programming, as with the data munging and video triggers.  However, having never used more than 4 LEDs to output recorded/live video, I couldn’t be so sure I could guarantee the display robustness – but with such a challenge, how could I say no to the project!After a few initial tests using a standard Arduino board in a non-stanard manner, I had managed to get ~20 LEDs lighting up with varying PWM values and we were off and running.  It turned out that the technique I had tested was naughtily using the arduinos’ onboard resources and was not a sustainable way of outputting video – so we had to look elsewhere.Options included using a daisy-chain of chips to multiply the output of an arduino duemillanove board, an arduino Mega and the phidgetLED 64.

With project timelines fairly short, we opted for the output mode we felt would be simplest/most trusted/idiot proof, which our experience told us would be the phidgetLED 64.  The phidget range of interface kits are bread and butter for the interactivation studio, as well as my computational environments students, as well as being able to claim a dedicated output of 64 PWM leds per board – which meant that we could order 3 and end up with spare LED output pins.The face itself could then be split up into separate sections to be addressed individually by each Phidget board – the forehead, center and chin regions containing around ~60 pixels each.  This allowed us to divide up the phidget output coding into regions and simplify a bit of our output matrixing.  I’d spent some time earlier working with maxduino to get greyscale LED output from pixelated video (a matrix of 6 x 1 pixels!), and luckily I was able to put that patch to work with a little bit of scaling, upgrading to the required resolution.The first issue we came to was the phidget method of sending single line matrices to the phidgetLED64 from top-left pixel to bottom-right pixel.  Since we were not working with a rectangular screen, each row of pixel data had to be offset from the starting 0 point, yet still line up with the neighbouring rows.

See Also;http://vividsydney.com/ http://www.smartlightsydney.com/artists/barker-and-haeusler http://www.timeoutsydney.com.au/aroundtown/smart-light-sydney–vivid-sydney.aspx

Related Posts:

Filtration Fields

August 6, 2009 Projects 2 comments

Recently Joanne and I were given the opportunity to exhibit in the DAB Lab Research Gallery at UTS, in the Design, Architecture and Building faculty building, as an opportunity to refine and showcase our collective research into realtime responsive architectural environments.The filtration fields exhibition in the DAB Lab gallery was a realtime interactive installation using simple camera tracking to measure daily activity within the DAB courtyard.  The exhibition was as a prototype test for ideas on the overlap of surveillance information and participation in architecture by its’ inhabitants.  Our premise for the installation was that the architecture of the DAB Lab gallery and surrounding courtyard space would be given eyes and ears, a brain to consider and a mouth to speak its’ mind.  The exhibition space of filtration fields was, unlike all pieces held in the DAB Lab, not the space of the gallery itself but the outside world upon which it had a threshold.  The silent box would become an active element in the architecture of the courtyard, no longer only passively inviting people inside but actively seeking to make its opinions known.  The void space of the courtyard would act as a performance stage for the activities and life of the DAB, and the natural bookend to the void was an appropriately matching wall of glass facing the space of the gallery.

The DAB gallery sits nestled under the canopy of one side in the DAB courtyard, standing as a window into another world, a place of existence in the imagined mind of another.  All of our experiences in the DAB Lab gallery were of surprise and delight, the little gallery had observed us and prepared something appropriate to show.My initial thoughts for the piece revolved around an image I had imagined of the DAB Lab gallery space existing as a small part of a sensory system extending the fabric of the whole building – the glass wall fronting onto the courtyard was in fact the glass lens of a large and ever curious eye.  The rear wall of the gallery would be the retina upon which the useful information would be refracted and transferred for processing elsewhere.  Other senses of the building were to be placed in the surrounding architecture outside, remote senses (microphones as ears, light/temp/hum/vibration as skin) of a much larger organism.  Each of the senses would be dislocated but connected, each informing the other regarding the goings-on of people in the courtyard.As the project took shape, it became clear that the focus of the exhibition should not only be the ‘eye’ of the DAB, but rather the effort to interpret the overlay of many eyes, ears and other senses into information, all representing the happenings in the courtyard.  The focus of the exhibition is not the DAB Lab itself, but the affect it could have on the lives of people moving through the space in-between.  Each of the glass wall panels would form opposing viewpoints on the courtyard, illustrating different relationships between the viewer/participant and the data they created.  The concept of the DAB as being a semi-conscious entity gave us the notion of eyes (an overload of information, all visual and uninterpreted for meaning) and brains (filtered information, abstracted for patterns of activity).

More to come..

Related Posts: