Matlab code for control of a resonant scanning microscope

Posted on December 1, 2016 by P.T.R. Rupprecht

For control of resonant scanning 2P microscopes, my host lab uses a software that I have written in Matlab. Due to some coincidences, the software is based on Scanimage 4.2, a version developed few years ago for an interface with a Thorlabs scope and Thorlabs software (DLLs). I basically threw out the whole Thorlabs software parts, rewrote the core processing code, but kept the program structure and the look-and-feel (see a screenshot below: looks like Scanimage, but it isn’t). For anybody interested, I uploaded the code to Github on my Instrument Control repository. The program’s name is scanimageB, to make clear that it is based on scanimage, but different at the same time.

guiAs hardware, the system is based on an Alazar 9440 DAQ board for 80 MHz acquisition with 2+ channels, where I was inspired by Dario Ringach’s Scanbox blog. Everything apart from acquisition is done using NI DAQ 6321 boards as in the original scanimage 4.2. Those boards are the cheapest X series DAQ boards. Some more details on the design are in this paper.

The software does not aim to be any kind of competitor for scanimage, scanbox, helioscan, sciScan, MScan or others. I do not even want other labto use this software for their microscopes. Instead, I’m hoping that people will find code snippets in the repository that might be useful for their own projects.
The code is not fully self-explanatory, and some core features (data acquisition) are partly dependent on the Alazar source developmental kit (ATS-SDK), which is cheap, but not open software. But if you are interested in a specific microscope control problem, send me a message, so that I can point you to the relevant code snippet which I used to solve this particular problem. Just let me know below in the comments or via eMail —

Here are some of the more interesting sections of the software:

  • MEX/C-code that uses native windows threads in C for parallelization and speeding up processing inside of Matlab. I use it to convert the 80 million data points per second per channel into pictures of arbitrary binning. Most other 2P resonant scanning microscopes do this task on (expensive) FPGAs.
    .
  • In one of the main m-files, search for scanphaseAdjust(obj). This is an algorithm that I’m using for automated scan phase adjustment for bidirectional scanning. The implementation is not designed for speed, but it features sub-pixel precision alignment by very simple means.
    .
  • In another big Matlab file which I repurposed from something written by Thorlabs, you can find how I implemented the integration of the Alazar 9440 DAQ board into Matlab using Alazar’s SDK, e.g. in the function PRinitializeAlazar(obj). When I started, I did not find any Matlab code online for controling this board, so this might serve as a starting point for other people as well.
    .
  • If you want to use retriggerable tasks for X-Series NI DAQ boards, you can search for the key words Task( and retriggerable in this code. Retriggerable tasks are important to understand if you want to synchronize devices on a sub-microsecond timescale using NI DAQ boards. This code snippets will give you a good idea how this can be done using the open DABS library (a Matlab instrument control library written by the Scanimage programmers). It works basically as in Labview, but the code can be understood more easily afterwards and by others.

Precise synchronization and reliable fast triggering is – in my opinion – the most challenging part of writing a control software for resonant scanning microscopes. To this end, I’m using the internal memory of the programmable X series NI DAQ boards to overcome these fast timescales (thereby following Scanimage 4.2 and 5.0). But the complex interdependence of triggers for laser pulses, lines, frames, laser shutters and pockels cells, together with the synchronization of external hardware makes things complicated and difficult to debug. If you are facing similar challenges of implementing complex triggering tasks, I would be glad to point you to sample code or give you some hopefully helpful advice —

Posted in Imaging, Microscopy | Tagged , , | 3 Comments

Weblogs on circuit and cellular neuroscience

Posted on November 28, 2016 by P.T.R. Rupprecht

A couple of days ago, I discovered a list of neuroblog feeds managed by Neurocritic, covering almost 200 blogs in total. Out of those, I picked the blogs most relevant for circuit and cellular neuroscience. This excludes most blogs on cognitive neuroscience and fMRI studies, and also those that focus on reproducibility and publishing issues or on science career advice rather than science itself. I preferred blogs which are well-written and still active, and which cover more than the papers of the lab or person that is running the blog. I also included blogs that focus on techniques and neuroengineering (those are covered here).

I put letters in front to inform about some of the blogs’ contents: l is a blog run by a neuroscience lab. p discusses scientific papers (though not always in depth). c includes some focus on computational aspects of neuroscience. And b openly discusses not only research papers and technical stuff, but also big questions that a general public might find intriguing.

p Short and very accessible spotlights on recent research: https://spikesandbursts.wordpress.com/

l b  Neuwritewest is an ambitious project aiming at making neuroscience more accessible to a broader public. It features paper presentations, and interviews with renown neuroscientists (‘Neurotalk’): http://www.neuwritewest.org/

c p b  Lists of recent papers on computational neuroscience and related topics. Including discussion of big questions in neuroscience, by Romain Brette: http://romainbrette.fr

p c Critical and very detailed discussion of recent neuroscience papers, by Markus Meister: https://markusmeister.com/

l p  From the lab of Anne Churchland from CSHL. Good discussion of recent topics in neuroscience and journal club discussions of single papers: https://churchlandlab.org/

b  A blog by neuroscientist Anita Devineni about her work experience with fruit flies and about big questions and topics in neuroscience. The blog is nicely designed and very well written: http://www.brains-explained.com

l c p  Discussion of recent papers in computational neuroscience by the lab of Jonathan Pillow: http://pillowlab.wordpress.com/

l p  A blog dedicated to bringing up and sometimes also discussing paper (mainly preprints posted on ArXiv and bioRxiv), run by the Steve Shea from CSHL: https://idealobserverblog.wordpress.com/

p  A blog discussing important papers with a focus on the hippocampus, run by Jake Jordan, a neuroscience PhD student in NY: https://nervoustalk.wordpress.com/

c p  Frequently updated lists of recent neuro-papers, although without any discussion: http://compneuropapers.tumblr.com/

p  Diverse blog posts with paper lists, some fun facts and neuroscience, run by Adam Calhoun: https://neuroecology.wordpress.com/

c p  Discussion of papers and topics, ranging from AI over cellular neuroscience to science politics. However, not updated recently: http://neurodudes.com/

Posted in Links, Neuronal activity | Tagged , | 1 Comment

Deep learning, part IV (2): Compressing the dynamic range in raw audio signals

Posted on October 5, 2016 by P.T.R. Rupprecht

In a recent blog post about deep learning based on raw audio waveforms, I showed what effect a naive linear dynamic range compression from 16 bit (65536 possible values) to 8 bit (256 possible values) has on audio quality: Overall perceived quality is low, mostly because silence and quiet parts of the audio signal will get squished. The Wavenet network by Deepmind, however, uses a non-linear compression of the audio amplitude that allowed to map the signal to 8 bit without major losses. In the next few lines, I will describe what this non-linear compression is, and how well it performs on real music.

Continue reading →

Posted in machine learning | Tagged , , | 4 Comments

Preamplifier bandwidth & two ways of counting photons

Posted on September 20, 2016 by P.T.R. Rupprecht

For two-photon point scanning microscopy, the excitation laser is typically pulsing at a repetition rate of 80 MHz, that is one pulse each 12.5 ns. To avoid aliasing, it was suggested to synchronize the sampling clock to laser pulses. For this, it is important to know over how much time the signal is smeared, that is, to measure the duration of the transient.

The device that smooths the PMT signal over time is the current-to-voltage amplifier. As far as I know, the two most commonly used ones are the Femto DHPCA-100 (variable gain, although mostly used with the 80 MHz bandwidth setting) and the Thorlabs model (60 MHz fixed bandwidth).

Observing single transients for different preamplifiers

However, 80 MHz bandwidth does not mean that everything below 80 MHz is transmitted and everything beyond suppressed. The companies provied frequency response curves, but in order to get a better feeling, I measured the transients of the above-mentioned preamplifiers when they amplified a single photon detected by a PMT. All transients are rescaled in y-direction (left-hand plot). I also determined a sort of gain for the single events by measuring the amplitude (right-hand plot). I also used two preamplifiers for each model, but could not make out any performance difference between two of the same kind.

preamps.png

For the 80 MHz bandwidth setting (Femto), the transient does not fully decay even after 15 ns or later; the Thorlabs preamp is even slower than this. Both exhibit a smooth multi-step shape during the decay phase.

At first glance, the Thorlabs preamp seems to be the obvious best choice, since the bandwidth is similar to the Femto 80 MHz and the gain is 3x higher. But for functional imaging, electrical noise is not a big problem, since the main source of noise is simply photon shot noise. In my hands, neither of both clearly outperformed the other, although this might be different for a completely different set of PMT/fluorophor/SNR.

The 200 MHz Femto setting would be perfect for lock-in sampling (and I already used it for that purpose), but the gain is, at least for my PMT, at the limit where electrical noise can become dominant (see figure on the right side). The 15 MHz setting, on the other hand, does not give any advantage, except if one samples with much less than 80 MHz.

Counting photons using an oscilloscope vs. using Poisson statistics

Looking at the raw oscilloscope traces when the microscope is scanning a biological sample, one can make another interesting observation. Shown here is a time window spanning 4000 ns, that is, 320 laser pulses. But those laser pulses only manage to elicit 14 photon-detecting events.

traces

Continue reading →

Posted in Calcium Imaging, Imaging, Microscopy | Tagged , , , | 10 Comments

Deep learning, part IV: Deep dreams of music, based on dilated causal convolutions

Posted on September 13, 2016 by P.T.R. Rupprecht

As many neuroscientists, I’m also interested in artificial neural networks and am curious about deep learning networks. I want to dedicate some blog posts to this topic, in order to 1) approach deep learning from the stupid neuroscientist’s perspective and 2) to get a feeling of what deep networks can and can not do. Part I, Part II, Part III, Part IVb.

One of the most fascinating outcomes of the deep networks has been the ability of the deep networks to create ‘sensory’ input based on internal representations of learnt concepts. (I’ve written about this topic before.) I was wondering why nobody tried to transfer the deep dreams concept from image creation to audio hallucinations. Sure, there are some efforts (e.g. this python project; the Google project Magenta, based on Tensorflow and also on Github; or these LSTM blues networks from 2002). But to my knowledge no one had really tried to apply convolutional deep networks on raw music data.

Therefore I downsampled my classical piano library (44 kHz) by a factor of 7 in time (still good enough to preserve the musical structure) and cut it into some 10’000 fragments of 10 sec, which yields musical pieces each with 63’000 data points – this is slightly fewer datapoints than are contained by 256^2 px images, which are commonly used as training material for deep convolutional networks. So I thought this could work as well. However, I did not manage to make my deep convolutional network classify any of my data (e.g., to decide whether a sample was Schubert or Bach), nor did the network manage to dream creatively of music. As most often with deep learning, I did not know the reasons why my network failed.

Now, Google Deepmind has published a paper that is focusing on a text-to-speech system based on a deep learning architecture. But it can also be trained using music samples, in order to lateron make the system ‘dream’ of music. In the deepmind blog entry you can listen to some 10 sec examples (scroll down to the bottom).

As key to their project, they used not only convolutional filters, but so-called dilated convolutions, thereby being able to span more length-(that is: time-)scales with fewer layers – this really makes sense to me and explains to some extent why I did not get anything with my normal 1d convolutions. (Other reasons why Deepmind’s net performs much better include more computational power, feedforward shortcut connections, non-linear mapping of the 16bit-resolved audio to 8bit for training and possibly other things.)

The authors also mention that it is important to generate the text/music sequence point by point using a causal cut-off for the convolutional filter. This is intuitively less clear to me. I would have expected that musical structure at a certain point in time could very well be determined also by future musical sequences. But who knows what happens in these complex networks and how convergence to a solution looks like.

Another remarkable point is the short memory of the musical hallucinations linked above. After 1-3 seconds, a musical idea is faded because of the exponential decaying memory; a bigger structure is therefore missing. This can very likely be solved by using networks with dilated convolutions that span 10-100x longer timescales and by subsampling the input data (they apparently did not do it for their model, probably because they wanted to generate naturalistic speech, and not long-term musical structure). With increasing computational power, these problems should be overcome soon. Putting all this together, it seems very likely that in 10 years you can feed the full Bach piano recordings into a deep network, and it will start composing like Bach afterwards, probably better than any human. Or, similar to algorithms for paintings, it will be possible to input a piano piece written by Bach and let a network which has learned different musical styles continuously transform it into Jazz.

On a different note, I was not really surprised to see some sort of convolutional networks excel at hallucinating musical structure (since convolutional filters are designed to interpret structure), but I am surprised to see that they seem to outperform recurrent networks for generation of natural language (this comparison is made in Deepmind’s paper). Long short-term memory recurrent networks (LSTM RNNs, described e.g. on Colah’s blog, invented by Hochreiter & Schmidhuber in ’97) solve the problem of fast-forgetting that is immanent to regular recurrent neuronal networks. I find it a bit disappointing that these problems can also be overcome by blown-up dilated convolutional feed-forward networks, instead of neuron-intrinsic (more or less) intelligent memory in a recurrent network like in LSTMs. The reason for my disappointment is due the fact that recurrent networks seem to be more abundant in biological brains (although this is not 100% certain), and I would like to see research in machine learning and neuronal networks also focus on those networks. But let’s see what happens next.

Continue reading →

Posted in machine learning | Tagged , , , | 8 Comments

Whole-cell patch clamp, part 1: introductory reading

Posted on September 4, 2016 by P.T.R. Rupprecht

Ever since I my interested in neuroscience become more serious, I was fascinated by the patch clamp technique, especially applied for the whole cell. Calcium imaging or multi-channel electrophysiology (recent review) is the way to go in order to get an idea what a neuronal population is doing on the single-cell level, but it occludes fast dynamics like bursting, fast oscillations and subthreshold membrane potential dynamics (calcium imaging), or unambiguous assignment of activity to single neurons (multi-channel ephys). That’s exactly what whole-cell patch clamp can do (and much more).

Some months ago, I started using the technique on an adult zebrafish brain ex vivo preparation. This image shows a z-stack of a patched cell that was imaged after the electrical recording. The surrounding cells are labeled with GCaMP; the brighter labeling of the patched neuron was done by a fluorophor inside the pipette that was diffusing into the cell, with which the pipette ideally forms a single electrical compartment. The fluorophor fills up the soma and some of the dendrites. The pipette position is shown as an overlay in the right-hand side image.

Electrophysiology is a very unrewarding and difficult activity, compared to calcium imaging. The typical, old-school electrophysiologist is always alone with his rig, through long nights of a never-ending series of failures, intercepted by few successfully patched and nicely behaving neurons. On average, frustration dominates, no matter how successful he/she is in the end; as a consequence, he fiercely protects his rig from anybody else who wants to touch it and might interfere with the labile stability of his setup. Therefore, over time, he becomes more and more annoyed by any interaction with fellow humans. At least that is what people say about electrophysiologists …

Despite this asocial component, nothing is more encouraging for beginners like me than hearing from others and about their struggles with electrophysiology. I will therefore write about my own experience with electrophysiology so far, and although I’m lacking the year-long experience of older electrophysiologist, I share my experience with the hope to encourage others.

To begin with, here’s a list of useful books and manuals for learning, if one does not have an experienced colleague who shows every single detail:

  • Areles Molleman, Patch Clamping: An Introductory Guide to Patch Clamp Electrophysiology
    A very short book which does not go into the details e.g. of analog electrical circuits of a cell, but gives useful pragmatic advice and how-to-dos for patching (both single channel and whole-cell). Very useful starting point for the beginner.
    .
  • In Labtimes, there’s a 2009 short first-hand report [the website is no longer online, but you can access it with the waybackmachine link] by Steven Buckingham that highlights some of the difficulties of patching and gives precise and concise advice.
    .
  • The Axon Guide for Electrophysiology & Biophysics Laboratory Techniques
    If you have time for 250 pages of technical descriptions, this is your choice. The document might be quite old, but there haven’t been many revolutions to patching anyway. For several troubleshooting issues, I have found good advice in this document.
    .
  • If you are lacking the theoretical background of how neurons, membrane potentials and ions work together, I would recommend online lectures like these slides that have a focus on theoretical underpinnings of measurements and not on measurements and troubleshooting.
    .
  • For a more in-depth description of everything related to membrane potentials and ions: Ion Channels of Excitable Membranes (3rd Ed.) by Bertil Hille. It’s 15 years old, but still the best book that I’ve seen so far. Especially for somebody with a physics background, it is very rewarding to read.
    .
  • For questions related to applications of patching (and other single neuron-specific tools), I can recommend Dendrites  by Stuart, Spruston, Häusser et al., although I have not yet checked the newest, very recent edition (2016)..

Soon, I hope that I will have time to write about some more technical aspects of patching. (Here about how to remove line-frequency noise that stems from the perfusion pump, about the limitations of quantitative whole cell voltage-clamp recordings, especially in small neurons, about blue light-induced artifacts in whole-cell recordings, and about the look and feel of two-photon targeted patching, including several instructive movies.)

Posted in electrophysiology, Microscopy, zebrafish | Tagged , , , | 2 Comments

The larval zebrafish, and the adult zebrafish

Posted on July 17, 2016 by P.T.R. Rupprecht

Zebrafish are often used as a model organism for in vivo brain imaging, because they are transparent. Or at least that is what many people think who do not work with zebrafish. In reality, most people use zebrafish larvae for in vivo imaging, typically not older than 5 days (post fertilization). At this developmental stage, the total larval body length is still less than the brain size of the adult fish. After 3-4 weeks, the fish look less like tadpoles and more like fish, measuring 10-15 mm in size (see also video below). They attain the full body length of approx. 25 to 45 mm within 3-4 months.

This video shows a zebrafish larva (7 days old), two adult zebrafish (16 months old) and a juvenile zebrafish (4.5 weeks old).

.
After 4-5 days, the brain size of larvae exceeds the dimensions that can be imaged with cellular resolution in vivo using light sheet or confocal microscopy when embedded in agarose. After approx. 4 weeks, even for unpigmented fish the thickened skull makes imaging of deeper brain regions very difficult. Superficial brain regions like the tectum are better accessible, but fish of this age are too strong to be restrained by agarose embedding. Brain imaging for adult fish is still possible in ex vivo whole brain preparations [1], but with loss of behavioral readout. Use of toxins for immobilization is an option (e.g. with curare in zebrafish [2] or in other fish species [3]), but not a legal one in some countries, including Switzerland. These are some of the reasons why most people stick to the simple zebrafish larva. My PhD lab is one of the few that does physiology in adult zebrafish.

Posted in Calcium Imaging, Neuronal activity, zebrafish | Tagged , | Leave a comment

Neuroscience on Youtube

Posted on July 15, 2016 by P.T.R. Rupprecht

Recently, I’ve been to the Basel ICON conference, where the recent Nobel laureate Eric Betzig gave an impressive talk on microscopy techniques (including lattice light sheet, SIM and expansion microscopy). Some days ago, I found a similar talk by Eric Betzig (although with less recent results) simply on youtube.
The advantages of online videos compared to live talks are obvious, and I wonder why people do not use them more often, both to learn about research and to communicate their own research. Additionally, compared to research papers, the personality of a researcher is much more obvious from a talk – which is important to know for students interested in working in his/her lab. Here just a small collection of some good talks by neuroscientists which are not as flashy and fancy as TED talks, but much more informative and interesting.

Here is Eve Marder on central pattern generators in lobsters and crabs. She had been developing experiments and models for this seemingly simply system for more than 40 years, thereby exposing the complexity of a system consisting of only 30 neurons.

Ken Harris, a mathematician by training and now more interested in large-scale brain activity recordings in mice, gives a rather technical, but very understandable talk on advances and problems in spike sorting for multielectrode arrays.

Larry Abbott, one of the most well-known theoreticians in neuroscience, with a very interesting talk about experimental findings in the olfactory system of the fruit fly.

Christof Koch on the search for the neuronal correlates of consciousness. In the late 90s, he was one of the pioneers in this field together with Francis Crick; more recently, he is working together with Giulio Tononi.

Edvard Moser on spatial navigation and place cells/grid cells. For this topic he was awarded the Nobel prize 2014.

Haim Sompolinsky with a theoretical perspective on sensory representations memory in distributed circuits. Coming from physics, Haim Sompolinsky helped transferring the physics of phase transitions to the mathematical modeling of neuronal network models in the late 80s.

Some of the links might be outdated in a couple of years, but I hope that researchers will start uploading more recent and well-prepared talks in the years to come, replacing overcrowded plenary talks by often jet-lagged speakers.

Update [2016-07-20]: Maybe this is the right place to mention a very nice series of podcasts, featuring interviews with leading neuroscientists, e.g. Michael Shadlen or Peter Jonas, or of my thesis supervisor in Basel, Rainer Friedrich. Thanks to Anne Urai who posted a link to this webpage on her blog.

Posted in Data analysis, Microscopy, Network analysis | Tagged , | Leave a comment

Large field of view microscopes for mouse brain imaging

Posted on June 30, 2016 by P.T.R. Rupprecht

For typical confocal or two-photon microscopes that maintain (sub)cellular resolution, a high-magnification objective is needed (typically 16x, 20x or 25x). This in turn limits the field of view (FOV) to ⌀ 1.0-1.5 mm.

For imaging in the mouse brain cortex, which is basically a big unwrinkled surface of a size of the order of 10 mm, a bigger FOV would be nice to have for some applications. Recently, a couple of papers came out that tried to increase the FOV, while using optical engineering to maintain the resolution. (Please don’t hesitate to tell me if I missed a relevant publication.)

Few years ago, I would have expected such papers to be published in Nature Methods, but apparently the time has come where optical engineering and improvement of existing techniques is not considered enough for passing the novelty bar. However, the three papers offer some very interesting lessons on engineering a two-photon microscope, of which I want to pick a few:

Continue reading →

Posted in Calcium Imaging, Imaging, Microscopy | Tagged , , | 7 Comments

Spatial visualization of temporal components for neuronal activity imaging

Posted on May 3, 2016 by P.T.R. Rupprecht

The standard analysis workflow for neuronal activity imaging based on calcium signals is to 1) draw ROIs around putative neurons, 2) extract the average fluorescence time trace of this ROI, 3) work with this timetrace for subsequent analysis (principal components, correlations between neuronal timetraces, tuning, etc).

For spatial visualization, typically a dF/F map is used, based on a F0 image and a time window used for computation of the dF map, showing the neurons active during a given time window. Here an example for one plane of the movie shown in a previous blog post.

anatomy_dF

I have rarely seen people mapping more complex temporal components back to space, although this is really easy … here I want to give an example, which I have used in a very similar way to create Fig.5D in this paper. (I have mentioned this paper earlier on this blog.) Here, I want to apply it to one plane of a small-volume recording.

First, I use the timecourses of single neuronal soma ROIs to calculate the temporal principal components of the time course of this set of neurons; or, often better, to generate clusters and the mean timetrace of each cluster. Here are the first three temporal clusters, showing three different dynamic features:

Traces

Next, I go back to the raw data and check each pixel for the correlation of the pixel’s time course with the time course of the components shown above. This gives one value for each pixel, i.e., in total a correlation map. This representation maps the temporal component back to space for all the three principal components that I’ve shown above. Here, the left picture corresponds to the red trace, the center picture to the green trace, the right picutre to the blue trace above.

RGB_gray

It is possible to condense this representation even further, by mapping each of the three spatial maps to one of the Red/Green/Blue color channels. In total, this is a spatial color-mapping of temporal components back to anatomy. I won’t discuss the biological interpretations here, but I find this data representation both appealing (although I’m partially colorblind) and helpful for understanding spatial structures.

RGB

Although being partially colorblind, I really like this color-coded spatial map. The arrow points to an artifact not related to neuronal activity, but a blood cell moving through a blood vessel during imaging.

The Matlab code behind this spatial mapping is very simple. Assuming you have the raw data (“movie”) and three extracted components (“clusters”):

movie; % 3D raw data, NxMxT (width x height x timepoints)
clusters; % extracted components TxW (T = timepoints, W = [1,2,3])

% 1 = R, 2 = G, 3 = B
for k = 1:3
   xcorr_tot = zeros(size(movie,1),size(movie,2));
   for j = 1:size(movie,1)
      xcorr_tot(j,:) = corr(clusters(:,k),squeeze(movie(j,:,:))');
   end
   RGB_map{k} = xcorr_tot;
end
Posted in Calcium Imaging, Data analysis, Imaging, Neuronal activity | Tagged , | Leave a comment