Key Research Findings

Simultaneous multi-scale electrophysiological measurement and optical manipulation of in vivo cortical networks

Key Collaborators: Peter Ledochowitsch (AIBS), Michel Maharbiz (UCB), Eddie Chang (UCSF), Andrea Hasenstaub (UCSF), Christoph Schreiner (UCSF)

 3DMultiscaleWe pioneered a 3D electrophysiological recording system that combines micro-eletrocorticography (uECoG) to record neural activity from the cortical surface over extended areas with mesoscale spatial resolution, with laminar polytrodes to densely record neural activity across cortical layers with microscale spatial resolution. Combining these high-temporal resolution, multiscale electrophysiological recordings with optical manipulations of neural activity further allows causal inference into the role of specific neural populations in local and distributed cortical computations. Examination of recorded signals >1kHz reveals neural events with timing and amplitude characteristics indicative of multi-unit action potentials evoked by sounds. Direct recordings of action potentials with laminar polytrodes inserted through perforations in the uECoG array suggest that functional tuning derived from uECoG (70-170Hz) reflects a spatial average of multiunit spiking activity immediately beneath the uECoG contacts. Focusing on the activity in the high-gamma range, we demonstrated that uECoG recorded field potentials have sufficient spatial resolution and selectivity to derive functional organization of rat auditory cortex (tonotopy), and thus provide a method for rapid, non-destructive mapping of cortical function. Preliminary results demonstrate the ability of uECoG to record neural changes in sound-evoked neural activity with high temporal resolution during optical manipulation of specific neuronal populations. Together, these results demonstrate high-temporal resolution, multi-scale electrophysiological measurements with simultaneous optical manipulation of in vivo cortical networks.

Related Publications:

Strategies for optical control and simultaneous electrical readout of extended cortical circuits

Ledochowitsch, P.*, Yazdan-Shahmorad, A.*, Bouchard, K.E., Diaz-Botia, C., Hanson, T., He, J., Seybold, B., Olivero, E., Blanche, T.J., Schreiner, C.E., Hasenstaub, A., Chang, E.F., Sabes, P.**, Maharbiz, M.M.**; J. Neurosci. Methods, July, 2015.

 

 

Statistical Methods for Consistent Selection and Unbiased Estimation of Model Parameters Key Collaborators: Sharmodeep Bhattacharyya (UCB)

A central goal of neuroscience is to understand how activity in the nervous system is related to features of the external world, or to features of the nervous system itself. A common approach is to model neural responses as a weighted combination of external features, or vice versa. The structure of the model weights can provide insight into neural representations. Often, neural input-output relationships are sparse, with only a few inputs contributing to the output. In part to account for such sparsity, structured regularizers are incorporated into model fitting optimization. However, by imposing priors, structured regularizers can make it difficult to interpret learned model parameters. The problem of accurately recovering model parameters from (noisy) data can be decomposed into selection of non-zero parameters and estimation of the values of those non-zero parameters. We combined previously separate methods for the selection and estimation problems. Specifically, we utilized BoLasso, debiased Lasso, and adaptive threshold selection methods to select parameters, followed by bagging ordinary least-squares estimates of the resulting selected parameters. Through extensive numerical investigations, we show that these methods perform favorably compared to previous approaches. In particular, for a variety of model distributions and noise levels, our methods more accurately recover the parameters of sparse models, leading to more parsimonious explanations of outputs, and higher predictive capacity on out-of-training data.

Related Publications:

Bootstrapped Adaptive Threshold Selection for Statistical Model Selection and Estimation

Bouchard, K.E.; arXiv.stat.ML; April, 2015.

 

Map of Vocal Tract Articulators in Human Sensorimotor Cortex During Speech Production Key Collaborators: Eddie Chang (UCSF)

SomatotopyHuman speech sounds (e.g. ‘shee’; far left plot) are generated by the rapid, coordinated movement of the vocal tract articulators (i.e. Lips, Jaw, Tongue, and Larynx; center left plot). We used high-density electrocorticography (ECoG) arrays to record neural activity directly from the surface of speech sensorimotor cortex (center right plot) while neurosurgical patients spoke consonant-vowel syllables. Analysis of neural activity revealed that speech articulators are represented in distinct spatial locations (far right plot). The relative location of the different representations were ordered as Larynx, Tongue, Jaw, Lips, Larynx. This was the first time all vocal tract articulators were simultaneously mapped on the human brain during the act of speaking.

Related Publications: 

Functional organization of human sensorimotor cortex for speech articulation

Bouchard, K.E., Mesgarani, N., Johnson, K., Chang, E.F.; Nature Article, Feb., 2013.

 

Context Dependent Cortical Network Dynamics for Syllable Production

Key Collaborators: Eddie Chang (UCSF)

CV_Cntxt_DynSpeech is the quintessential sequential behavior. The cortical dynamics generating these sequences is largely unknown. We used dimensionality reduction (e.g. PCA, LDA) to map the high-dimensional network activity (ECoG data) to a lower dimensional space (‘cortical state-space’) that faithfully retains the structure of the entire network. We first examined state-space dynamics giving rise to different sets of consonant-vowel syllables (left plot). This revealed periodic dynamics during the network transition between distinct states for consonants and vowels. Consonants and vowels were further sub-specified for consonant and vowel features. The organization of phonetic features in the state-space emphasized the articulatory differences between phonemes, but contained additional structure. We then examined the detailed network dynamics with which single phonemes (e.g. /g/ and /u/) transitioned to/from multiple other phonemes on a single-trial basis (right plot). This revealed that the state-space trajectories for single phonemes are biased towards the state-space locations of adjacent phonemes. This bias effectively minimizes the distance the network travels between adjacent states, in agreement with predictions of optimal control theory.

Related Publications: 

Functional organization of human sensorimotor cortex for speech articulation

Bouchard, K.E., Mesgarani, N., Johnson, K., Chang, E.F.; Nature Article, Feb., 2013.

Cortical control of vowel formants and co-articulation by human sensorimotor cortex

Bouchard, K.E., Chang, E.F.; J. Neuroscience, Sept., 2014.

 

 

Towards Brain-Machine Interfaces: Single-trial Decoding of Produced Speech Key Collaborators: Eddie Chang (UCSF)

 Brain-machine interfaces (BMI) involve deriving a mathematical mapping from neural signals (brain) to the control of an external effector (machine). There are a large variety of clinical and basic science applications BMIs. However, current performance is relatively modest, especially for speech. Our studies are currently the state-of-the-art in single-trial decoding of speech from human sensorimotor cortex. Together, other results suggest a novel, mixed continuous/discrete approach to a speech prosthetic.

Dcd

Continuous Decoding of Vowel Acoustics. The speech sensorimotor cortex controls the kinematics of the vocal tract articulators. In general, the relationship between vocal tract kinematics and produced acoustics is a many-to-one mapping, and is thus a mathematically degenerate problem. However, the acoustics of vowels during the steady state is more directly related to acoustics than for most other sounds. Therefore, we utilized a novel approach to statistical regularization to decode the produced continuous acoustics of three cardinal vowels (/a/ (‘aa’), /i/ ‘ee’, /u/ ‘oo’) from the concurrently recorded neural activity (left plot). We were able to predict the acoustics of produced vowels on a single-trial basis with extremely high accuracy. This is currently the state-of-the-art in published continuous speech decoding.

Related Publications:

Neural decoding of spoken vowels from human sensory-motor cortex with high-density electrocorticography

Bouchard, K.E., Chang, E.F.; IEEE, EMBC, Aug., 2014.

Cortical control of vowel formants and co-articulation by human sensorimotor cortex

Bouchard, K.E., Chang, E.F.; J. Neuroscience, Sept., 2014.

 

Continuous Decoding of Articulator Kinematics. Although we have demonstrated the state-of-the-art in continuous decoding of vowel acoustics, our ability to decode the acoustics of other speech sounds, such as consonants is relatively poor. As discussed above, this in part reflects the fact that the sensorimotor cortex controls the vocal tract articulators, and the mapping from articulators to acoustics is, in general, degenerate. However, simultaneous measurement of all vocal tract articulators is challenging, especially in the clinical setting in which our ECoG recordings are taken. To overcome this challenge, we pioneered a novel, multi-modal system for real-time tracking of all vocal tract articulators during speech production compatible with ECoG recordings in the hospital. Preliminary analysis of lip kinematics has demonstrated the capacity to predict the lip aperature with high fidelity (center). This is the first time that vocal tract articulator kinematics have been directly decoded from human brain recordings.

Related Publications:

High-resolution, non-invasive imaging of upper vocal tract articulators compatible with human brain recordings

Bouchard, K.E.*, Conant, D.*, Anumanchipalli, G., Dichter, B.; Johnson, K., Chang, E.F.; PLoS One. 2016

 

Deep Neural Nets for Syllable Classification: In contrast to the continuous decoding described above, the most common approach to speech prostheses in the literature treats speech as a sequence of categorical tokens, and attempts to classify these tokens from the recorded neural activity. Most recently, we have been using deep neural networks to classify speech syllables (right). Our initial results are far surpassing the current state-of-the-art published results for speech classification. These results suggest that deep neural networks will be fruitful avenues for neural prosthetics.

Related Publications:

Decoding speech from human ECoG with deep networks

Livesey, J.*, Anumanchipalli, G.K.*, Prabhat, Bouchard, K.E.** Chang, E.F.**; NIPS Poster; June, 2015.

 **: co-senior authors

 

Latent Structure of Spatio-temporal Patterns of Neural Activity Revealed by DNNs Key Collaborators: Jesse Livezey (UCB), Eddie Chang (UCSF)

clust_dnn

Brain computations are non-linear functions operating on spatio-temporal patterns of neural activity. However, most methods used to understand brain computation are linear, inherently limiting the capacity to extract structure from neural recordings. This is an issue not only for optimization of BMIs, but also for understanding brain computations. As DNNs are essentially adaptive bases function, non-linear function approximators, it is possible that they can extract structure from noisy, single-trial neural recordings that reveal important organization of representations. We examined the structure of network output to more fully understand the organization of syllable representations in vSMC.To the left, we show the average confusion matrix resulting from the output of the softmax layer of the fully connected network (i.e. before binary classification), with target syllables arranged along rows and predicted syllable across columns. The syllables are ordered according to the results of agglomerative hierarchical clustering using Ward’s method. To the right is a bar-plot of the mean accuracy with which a specific syllable was correctly classified. Note that the syllable with worst accuracy is the one with the smallest number of examples in the dataset. At the highest level, syllables seem to be confused only within the articulator involved (lips, back tongue, or front tongue) in the syllable. This is followed by a characterization of the place of articulation within each articulator (bilabial, labio-dental, etc.). At the lowest level there seems to be a clustering across the vowel categories that capture the general shape of the vocal tract in producing the syllable. These results demonstrate the capacity of deep networks to reveal important structure in single-trial neural recordings that is not recoverable with other methods.

Related Publications:

Decoding speech from human ECoG with deep networks

Livesey, J.*, Anumanchipalli, G.K.*, Prabhat, Bouchard, K.E.** Chang, E.F.**; NIPS Poster; June, 2015.

 **: co-senior authors

 

Sparse Sensorimotor Representations of Articulators

Key Collaborators: Eddie Chang (UCSF) and Fritz Sommer (UCB)

VclTrct.ICntrl

In contrast to our understanding of motor systems, our understanding of sensory systems is relatively advanced. The concept of sparsity has proven central to a theoretical understanding of sensory processing. A population representation is said to be sparse if only a small number of elements are active at any moment in time. Mathematically, sparse representations are often statistically independent. We applied unsupervised ICA to the spatio-temporal patterns of neural activity associated with the different syllables in our data set, and simply visualized the resulting components. We observed two distinct types of sparse representations in sensorimotor cortex activity. Several ICs that accounted for the largest amount of variance were activated by all syllables that engaged the same articulator (left). Specifically, we found that IC2 was activated during syllables with labial consonants (green), IC4 was activated during syllables with dorsal tongue consonants, and IC6 was activated during syllables with coronal tongue consonants. This suggests a sparse representation in an articulator basis. Together, these data support the notion of sparse sensorimotor representations during behavior, and are the first observations of their type mammalian motor cortex.

Related Publications: 

Sparse coding of ECoG signals identifies interpretable components for speech control in human sensorimotor cortex

Bouchard, K.E.*, Bujan, A.F., Chang, E.F., Sommer, F.T.; In press, IEEE, EMBC, Aug., 2017.

 

Encoding and Integration of Learned Probabilistic Birdsong Sequences

Key Collaborators: Michael Brainard (UCSF, HHMI)

MBrd_prb_encdany complex behaviors, such as human speech and birdsong, reflect a set of categorical actions that can be flexibly organized into variable sequences.  However, little is known about how the brain encodes the probabilities of such sequences. Behavioral sequences are typically characterized by the probability of transitioning from a given action to any subsequent action (forward probability; left plot).  In contrast, we hypothesized that neural circuits might encode the probability of transitioning to a given action from any preceding action (backward probability; left plot). To test whether backward probability is encoded in the nervous system, we investigated how auditory-motor neurons in vocal premotor nucleus HVC of songbirds encode different probabilistic characterizations of produced syllable sequences. We recorded responses to auditory playback of pseudo-randomly sequenced syllables from the bird’s repertoire, and found that variations in responses to a given syllable could be explained by a positive linear dependence on the backward probability of preceding sequences (center plot). Furthermore, backward probability accounted for more response variation than other probabilistic characterizations, including forward probability (center plot).  Finally, we found that responses integrated over more than 7-10 syllables (~700-1000 ms) with the sign, gain and temporal extent of integration depending on backward probability (right plot). Our results demonstrate that backward probability is encoded in sensory-motor circuitry of the song-system, and suggest that encoding of backward probability is a general feature of sensory-motor circuits.

Related Publications:

Neural encoding and integration of learned probabilistic sequences in avian sensory-motor circuitry

Bouchard, K.E., Brainard, M.S.; J. Neuroscience, Nov., 2013.

  

Predictive neural dynamics for learned temporal and sequential statistics

Key Collaborators: Michael Brainard (UCSF, HHMI)

brd_prb_prdctPredicting future events is a critical computation for both perception and behavior. Despite the essential nature of this computation, there are few studies demonstrating neural activity that predicts specific events in learned, probabilistic sequences. Here, we test the hypotheses that the dynamics of internally generated neural activity are predictive of future events and are structured by the learned temporal-sequential statistics of those events. We recorded neural activity in Bengalese finch sensory-motor area HVC in response to playback of sequences from individuals’ songs, and examined the neural activity that continued after stimulus offset. We found that auditory responses to playback of syllable sequences are tuned to the produced timing between syllables. Furthermore, post-stimulus neural activity induced by sequence playback resembles neural responses to the next syllable in the sequence when that syllable is predictable, but not when the next syllable is uncertain. Our results demonstrate that the dynamics of internally generated HVC neural activity are predictive of the learned temporal-sequential structure of produced song and that the strength of this prediction is modulated by uncertainty.

Related Publications:

Auditory induced neural dynamics in sensory-motor circuitry predict learned temporal and sequential statistics of birdsong

Bouchard, K.E., Brainard, M.S.; PNAS, Aug., 2016.

 

 

Encoding of Learned Probabilistic Speech Sequences

Key Collaborators: Matt Leonard (UCSF) and Eddie Chang (UCSF)

Humn_phntct_encdSensory processing involves identification of stimulus features, but also integration with the surrounding sensory and cognitive context. Previous work in animals and humans has shown fine-scale sensitivity to context in the form of learned knowledge about the statistics of the sensory environment, including relative probabilities of discrete units in a stream of sequential auditory input. These statistics are a defining characteristic of one of the most important sequential signals humans encounter: speech. For speech, extensive exposure to a language tunes listeners to the statistics of sound sequences. To address how speech sequence statistics are neurally encoded, we used high-resolution direct cortical recordings from human superior temporal gyrus (right plot) as subjects listened to words and nonwords with varying transition probabilities between sound segments (center plot). In addition to their sensitivity to acoustic features (including contextual features like coarticulation), we found that neural responses dynamically encoded the language-level probability of both preceding and upcoming speech sounds. Transition probability first negatively modulated neural responses, followed by positive modulation of neural responses, consistent with coordinated predictive and retrospective recognition processes, respectively (right plot). These results demonstrate that sensory processing of deeply learned stimuli involves integrating physical stimulus features with their contextual sequential structure.

Related Publications:

Dynamic encoding of speech sequence probability in human temporal cortex

Leonard, M.K., Bouchard, K.E., Tang, C; Chang, E.F.; J. Neuroscience, May, 2015.

  

Hebbian Mechanisms of Probabilistic Sequence Encoding in Neural Networks Key Collaborators: Michael Brainard (UCSF)

 Prob_LearnThe majority of distinct motor (and sensory) events occur as temporally ordered sequences with rich probabilistic structure. Sequences can be characterized by the probability of transitioning from the current state to upcoming states (forward probability), as well as the probability of having transitioned to the current state from previous states (backward probability). Despite the prevalence of probabilistic sequencing of both sensory and motor events, the Hebbian mechanisms (left plot) that mold synapses to reflect the statistics of experienced probabilistic sequences are not well understood. We demonstrated that to stably reflect the conditional probability of a neuron’s inputs and outputs, local Hebbian plasticity requires a balance of the magnitude of competitive forces and homogenizing forces (center plot). We showed through analytic calculations and numerical simulations that asymmetric Hebbian plasticity (correlation, covariance and STDP) with pre-synaptic competition can develop synaptic weights equal to the conditional forward transition probabilities present in the input sequence (right plot). In contrast, post-synaptic competition can develop synaptic weights proportional to the conditional backward probabilities of the same input sequence (right plot). Together, these results demonstrate a simple correspondence between the biophysical organization of neurons, the site of synaptic competition, and the temporal flow of information encoded in synaptic weights by Hebbian plasticity while highlighting the necessity of balancing learning forces to accurately encode probability distributions.

Related Publications:

Role of the site of synaptic competition and the balance of learning forces for Hebbian encoding of probabilistic Markov sequences

Bouchard, K.E., Ganguly, S., Brainard, M.S.; Front. in Comp. Neuro., July, 2015.

 

32 pings

Skip to comment form

  1. A look at deep learning for science | Vedalgo

    […] recent work, we found that, even on the relatively small data sets typically attained in neuroscience […]

  2. A look at deep learning for science | A bunch of data

    […] recent work, we found that, even on the relatively small data sets typically attained in neuroscience […]

  3. A look at deep learning for science – Cloud Data Architect

    […] recent work, we found that, even on the relatively small data sets typically attained in neuroscience […]

  4. look at this site

    Related…

    […]just beneath, are numerous totally not related sites to ours, however, they are surely worth going over[…]…

  5. 科学领域中的深度学习一览

    […] 在最近的工作中,我们发现即使在通过神经科学实验获得的相对较小的数据集上,DNN方法优于传统的解码(即翻译)大脑信号产生语音的方法,达到了最先进的语音分类性能(高达39%的准确度,是随机瞎猜的25倍多)。此外随着训练数据集的增大,DNN的表现会比传统的分类器更好,其在相对有限但非常有价值的数据上实现了回报最大化。输入数据集包括85个频道和250个时间采样信号,分成了1到57个类别。一个单独主题的数据集通常只有2000个训练样本,需要大量的超参数搜索以得到最佳表现。最好的网络具有一个或两个具有双曲正切非线性的隐藏层,并使用Theano库在GPU和CPU上进行训练。每个模型训练相对较快(30分钟),但是很多模型已经在超参数搜索中训练好了。这些结果表明,DNN将来可能成为大脑-机器接口的最先进的方法,这需要更多的工作来找到在小型数据集上训练深度神经网络的最佳实践。 […]

  6. samsung power board

    Superb website

    […]always a big fan of linking to bloggers that I love but don’t get a lot of link love from[…]…

  7. Rajamangala University of Technology Thanyaburi

    Recent Blogroll Additions…

    […]usually posts some very interesting stuff like this. If you’re new to this site[…]…

  8. thc distillate

    Awesome website

    […]the time to read or visit the content or sites we have linked to below the[…]…

  9. Web Design Solution

    Sources

    […]check below, are some totally unrelated websites to ours, however, they are most trustworthy sources that we use[…]…

  10. 240V inverter

    Read was interesting, stay in touch…

    […]please visit the sites we follow, including this one, as it represents our picks from the web[…]…

  11. Life insurance quote

    Cool sites

    […]we came across a cool site that you might enjoy. Take a look if you want[…]…

  12. kala jadoo

    Websites you should visit

    […]below you’ll find the link to some sites that we think you should visit[…]…

  13. amil baba

    Cool sites

    […]we came across a cool site that you might enjoy. Take a look if you want[…]…

  14. what is a succubus

    Awesome website

    […]the time to read or visit the content or sites we have linked to below the[…]…

  15. airport black car

    You should check this out

    […] Wonderful story, reckoned we could combine a few unrelated data, nevertheless really worth taking a look, whoa did one learn about Mid East has got more problerms as well […]…

  16. Motorcycle insurance rates comparison

    Websites we think you should visit

    […]although websites we backlink to below are considerably not related to ours, we feel they are actually worth a go through, so have a look[…]…

  17. skuteczne tabletki na chrapanie

    Visitor recommendations

    […]one of our visitors recently recommended the following website[…]…

  18. The Storybook Kingdom

    Visitor recommendations

    […]one of our visitors recently recommended the following website[…]…

  19. blackjack

    Sources

    […]check below, are some totally unrelated websites to ours, however, they are most trustworthy sources that we use[…]…

  20. online blackjack oyna

    Great website

    […]we like to honor many other internet sites on the web, even if they aren’t linked to us, by linking to them. Under are some webpages worth checking out[…]…

  21. online casino siteleri

    Visitor recommendations

    […]one of our visitors recently recommended the following website[…]…

  22. bedava bahis

    Awesome website

    […]the time to read or visit the content or sites we have linked to below the[…]…

  23. online 21 oyna

    Online Article…

    […]The information mentioned in the article are some of the best available […]…

  24. 6oda.com

    Superb website

    […]always a big fan of linking to bloggers that I love but don’t get a lot of link love from[…]…

  25. medical tourism

    Sources

    […]check below, are some totally unrelated websites to ours, however, they are most trustworthy sources that we use[…]…

  26. absaroka guest house.net

    Sites we Like…

    […] Every once in a while we choose blogs that we read. Listed below are the latest sites that we choose […]…

  27. asamedyaajansi.com

    Cool sites

    […]we came across a cool site that you might enjoy. Take a look if you want[…]…

  28. adapastanesi.com

    Read was interesting, stay in touch…

    […]please visit the sites we follow, including this one, as it represents our picks from the web[…]…

  29. akyarbalik.web.tr

    Read was interesting, stay in touch…

    […]please visit the sites we follow, including this one, as it represents our picks from the web[…]…

  30. albatroshavacilik.com

    Recommeneded websites

    […]Here are some of the sites we recommend for our visitors[…]…

  31. alfa restaurant abant

    Visitor recommendations

    […]one of our visitors recently recommended the following website[…]…

  32. alfa butik otel

    Gems form the internet

    […]very few websites that happen to be detailed below, from our point of view are undoubtedly well worth checking out[…]…

Comments have been disabled.