oh, son of a bitch
May. 25th, 2019 11:00 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
sciatrixI was spending my day sitting down and going through all my playback acoustic data for my mice. This is my second chapter, I'm supposed to graduate in the spring, my PI keeps reminding me that I have to make progress, and... well, it's a pretty complicated thing I'm trying to do. I've had several errors that have set the work back by losing animals so far, and I'm very, very animal-limited: my singing mice only have litters that average ~2 or 3 pups per litter, and there are a number of experiments going on that are using the animals our colony does produce.
This experiment, I'm recording them for 48h, but the really crucial part is the 1h they spend listening to recordings and potentially responding. In previous recording paradigms, we manually hit "save" whenever the mics recorded, which was triggered only when a mouse sang and ran for about 15 seconds; in this one, I'm recording the whole time and trying to sift through massive amounts of data to find the relevant stuff. Even for just that super-relevant 1-hour interval, each recorded file is ~1.2 GB, and my poor little laptop can only handle so much. (The desktop I'm running the actual recording on is even more hampered; it's still on Windows 7 and hasn't been upgraded since I joined the lab in 2012.) The previous paradigm also sucked--my last experiment, about half the songs were clipped because the cue to start recording started so late--but at least I was a little more familiar with it.
This is the first time I've run an experiment with this paradigm (where "this" = "spring break over March"), after I spent months trying to figure out how to even do it, via Matlab using ActiveX controls to speak to the proprietary software that our acoustic processor runs on. On top of that, I'm still ironing out the kinks in the hardware, which is notoriously unreliable: it has, in the seven years I've been working with it, consistently up and died whenever I needed to start working with it again, and absolutely no one knows why. (The mics are also often taken out to the field, where they wind up being used in a humid environment and packed around and rummaged through by the TSA.) Each mic is a tiny little thumbnail-sized piece that is run through four amplifiers in sequence before being hooked up to our processor, so there are quite a number of places where something might go wrong, including at least three different connecting cables between the amps.
I have run through them all before, and I will be running through them again this weekend, labelmaker in hand, trying to see if I can get more functional systems together which I can completely trust to work.
Again, because I did all of this over the winter when I was trying to work out why things kept breaking.
I thought I'd solved a large part of the puzzle when I looked at the electrical circuits themselves and found they were overloaded: I figured that they might be routinely causing fluctuations in the power grid that were overloading the amps, especially the second amps from the microphone. So I talked to an electrician and tried to get that sorted out this winter, buuuut there's a freezer in the room that can't be moved and a whole lot of pretty heavy electrical equipment in there, and all I was able to do was cut several items and balance the rest as best I could across the room's two circuits. Oh, and run everything through a power conditioner, but I was pretty sure that previous attempts to deal with this via nested power conditioners had been making things worse.
At the time, I went through and checked each individual hardware component for function, but I only really went through and looked at the recordings of empty space in a sound chamber because the best I could do was minimize noise rather than eliminating it entirely. If it gave me a recording that looked more or less clean rather than being completely overwhelmed by noise, I called it good. I later did check out how my stimuli were being recorded on the readings, but... crucially... only in one of the four chambers. They all recorded okay with minimal noise when I checked previous recordings, and I was trying to set up animals to run in between teaching and dissecting out new tissues* and figuring out how to process the sheer magnitude of these recordings--seriously, trying to make a spectrogram of one of them crashed my laptop--and dealing with my committee meeting and grading finals and working out every other goddamn thing, not to mention all the hormonal research I've been doing on where to send the blood-- well, I figured it was fine, especially because I was glancing at what I could of the recordings. It's just I kept glancing at chamber 3 recordings, because they were the top of my list.
*I figured out the secret to removing the inguinal fat pad smoothly last week! It is unspeakably disgusting.
As it turns out, that was the only chamber that seems to have been recording anything but noise. At least, that's what I can figure out from looking at playout recordings from other chambers. As far as I can tell, the other three mics are recording variations on noise, but the low-frequency band of strong noise I have to filter out is still present. Previously, I'd lost trials down to: a) stimuli playouts that weren't sampled properly for the apparatus, b) misfiring timings of the triggers that play out. So this hits prettty hard, because I'm so close but I keep hitting new setbacks, and this isn't the first setback on this particular project. And the sound stuff is such a familiar setback, one that no one but me really knows how to roll with at this point.
Anyway, I have lost a truly staggering amount of the behavioral data I thought I had, and I will be spending a large part of tomorrow desperately tinkering with my hardware and trying to make the thing go before I contact my boss and ask to replace some component of hardware and/or explain that I've had yet. another. setback.
This experiment, I'm recording them for 48h, but the really crucial part is the 1h they spend listening to recordings and potentially responding. In previous recording paradigms, we manually hit "save" whenever the mics recorded, which was triggered only when a mouse sang and ran for about 15 seconds; in this one, I'm recording the whole time and trying to sift through massive amounts of data to find the relevant stuff. Even for just that super-relevant 1-hour interval, each recorded file is ~1.2 GB, and my poor little laptop can only handle so much. (The desktop I'm running the actual recording on is even more hampered; it's still on Windows 7 and hasn't been upgraded since I joined the lab in 2012.) The previous paradigm also sucked--my last experiment, about half the songs were clipped because the cue to start recording started so late--but at least I was a little more familiar with it.
This is the first time I've run an experiment with this paradigm (where "this" = "spring break over March"), after I spent months trying to figure out how to even do it, via Matlab using ActiveX controls to speak to the proprietary software that our acoustic processor runs on. On top of that, I'm still ironing out the kinks in the hardware, which is notoriously unreliable: it has, in the seven years I've been working with it, consistently up and died whenever I needed to start working with it again, and absolutely no one knows why. (The mics are also often taken out to the field, where they wind up being used in a humid environment and packed around and rummaged through by the TSA.) Each mic is a tiny little thumbnail-sized piece that is run through four amplifiers in sequence before being hooked up to our processor, so there are quite a number of places where something might go wrong, including at least three different connecting cables between the amps.
I have run through them all before, and I will be running through them again this weekend, labelmaker in hand, trying to see if I can get more functional systems together which I can completely trust to work.
Again, because I did all of this over the winter when I was trying to work out why things kept breaking.
I thought I'd solved a large part of the puzzle when I looked at the electrical circuits themselves and found they were overloaded: I figured that they might be routinely causing fluctuations in the power grid that were overloading the amps, especially the second amps from the microphone. So I talked to an electrician and tried to get that sorted out this winter, buuuut there's a freezer in the room that can't be moved and a whole lot of pretty heavy electrical equipment in there, and all I was able to do was cut several items and balance the rest as best I could across the room's two circuits. Oh, and run everything through a power conditioner, but I was pretty sure that previous attempts to deal with this via nested power conditioners had been making things worse.
At the time, I went through and checked each individual hardware component for function, but I only really went through and looked at the recordings of empty space in a sound chamber because the best I could do was minimize noise rather than eliminating it entirely. If it gave me a recording that looked more or less clean rather than being completely overwhelmed by noise, I called it good. I later did check out how my stimuli were being recorded on the readings, but... crucially... only in one of the four chambers. They all recorded okay with minimal noise when I checked previous recordings, and I was trying to set up animals to run in between teaching and dissecting out new tissues* and figuring out how to process the sheer magnitude of these recordings--seriously, trying to make a spectrogram of one of them crashed my laptop--and dealing with my committee meeting and grading finals and working out every other goddamn thing, not to mention all the hormonal research I've been doing on where to send the blood-- well, I figured it was fine, especially because I was glancing at what I could of the recordings. It's just I kept glancing at chamber 3 recordings, because they were the top of my list.
*I figured out the secret to removing the inguinal fat pad smoothly last week! It is unspeakably disgusting.
As it turns out, that was the only chamber that seems to have been recording anything but noise. At least, that's what I can figure out from looking at playout recordings from other chambers. As far as I can tell, the other three mics are recording variations on noise, but the low-frequency band of strong noise I have to filter out is still present. Previously, I'd lost trials down to: a) stimuli playouts that weren't sampled properly for the apparatus, b) misfiring timings of the triggers that play out. So this hits prettty hard, because I'm so close but I keep hitting new setbacks, and this isn't the first setback on this particular project. And the sound stuff is such a familiar setback, one that no one but me really knows how to roll with at this point.
Anyway, I have lost a truly staggering amount of the behavioral data I thought I had, and I will be spending a large part of tomorrow desperately tinkering with my hardware and trying to make the thing go before I contact my boss and ask to replace some component of hardware and/or explain that I've had yet. another. setback.
no subject
Date: 2019-05-26 04:07 am (UTC)no subject
Date: 2019-05-26 09:51 pm (UTC)And then on top of that, my boss is an inexperienced mentor (definition: has graduated I think... four or five students? had graduated exactly one when I joined the lab) who means well but is also definitely not great at his job, if you define a PhD advisor's job as providing a structure to get students out in a timely manner, and is working in a system that no one has worked in but himself since the 1970s, and I've been carrying the brunt of a lot of that...
well, it's perhaps not surprising it's going to take me eight years.
no subject
Date: 2019-05-26 07:20 am (UTC)(I once had a temperature logger silently fail for several runs of an experiment. When temperate was one of the major variables. And I had very limited time with equipment. That was... fun.)
no subject
Date: 2019-05-26 09:39 pm (UTC)Silent failure would have been an improvement.
no subject
Date: 2019-05-26 09:03 am (UTC)no subject
Date: 2019-05-26 09:39 pm (UTC)no subject
Date: 2019-05-26 10:35 am (UTC)no subject
Date: 2019-05-26 09:39 pm (UTC)no subject
Date: 2019-05-26 04:42 pm (UTC)Reminds me of a postgrad here who learned the hard way that mp3's were silently excluded from institutional backups because they're clearly just somebody's music collection (except when they're not).
no subject
Date: 2019-05-26 06:15 pm (UTC)that's uh, absolutely horrifying. I don't keep my recordings in mp3 format--they're all .float32 files--but I can imagine that happening and it's so so bad.