A novel experimental and analytical approach to the multimodal neural decoding of intent during social interaction in freely-behaving human infants 7m database soccer basketball
Understanding typical and atypical development remains one of the fundamental questions in developmental human neuroscience. S cerevisiae database Traditionally, experimental paradigms and analysis tools have been limited to constrained laboratory tasks and contexts due to technical limitations imposed by the available set of measuring and analysis techniques and the age of the subjects. Mode s database These limitations severely limit the study of developmental neural dynamics and associated neural networks engaged in cognition, perception and action in infants performing “in action and in context”. Sybase database This protocol presents a novel approach to study infants and young children as they freely organize their own behavior, and its consequences in a complex, partly unpredictable and highly dynamic environment. Database types The proposed methodology integrates synchronized high-density active scalp electroencephalography (EEG), inertial measurement units (IMUs), video recording and behavioral analysis to capture brain activity and movement non-invasively in freely-behaving infants. Database testing This setup allows for the study of neural network dynamics in the developing brain, in action and context, as these networks are recruited during goal-oriented, exploration and social interaction tasks.
One of the fundamental human capabilities is the ability to learn and deploy actions (action production) strategically in service of goals and rewards, the ability to apprehend the goals of social partners (action understanding) in order to produce appropriate social responses, and the ability to learn from others through observation and imitation 1. Database tutorial The neural basis of these cognitive-motor capabilities have been attributed, at least in part, to the so-called mirror neuron system; a system that is thought to be engaged when one views someone performing an action and when one performs the action.
Database transaction However, the potential link between the mirror neuron system and action understanding is not yet well understood 1. Database triggers Studying the emergence and development of this mirror neuron system in human infants has been hampered by a) the technical limitations of multi-modal data acquisition of brain activity correlated to intent and fine-grained motion data, b) the constraints imposed by experimental protocols that are unnatural ( e.g., social interaction with an agent depicted in a videotape, the need for maintaining a still posture to minimize artifacts during Electroencephalographic (EEG) recordings, etc.), and c) the communication/language barriers when testing young infants/toddlers that greatly limit the researcher’s capabilities to give instructions and validate behaviors.
For a better understanding of the varying neural and behavioral dynamics in natural behavior, we developed a novel experimental and analytical approach that allows the time-resolved study of the neural substrates of emerging goal-oriented and social behaviors in young children. Database tools Specifically, we deployed an EEG based mobile brain imaging (MoBI) approach 2 to record brain activity and movement from freely-behaving infants during interaction with an experimenter. Database technology Inertial measurement units (IMUs) were used to monitor subject and experimenter’s kinematics.
EEG technology and inertial sensors were used to study neural patterns and activations associated to the infants’ action imitation and goal-oriented behaviors in an unscripted interaction with an experimenter/actor. Database terminology Actions such as reach-grasp, reach-offer, observe, rest, and explore are all part of the cognitive-motor processes involved in imitation. Database training Furthermore, we use source estimation to localize the generators of electric potentials within the brain during the behavioral tasks, thereby studying the spatiotemporal dynamics of neuronal currents throughout the brain. Database theory Similarly we deploy machine learning algorithms to assess and measure the predictability of these behavioral actions by identifying action-relevant spatio-temporal patterns in the neural activity in sensor (EEG) and/or source spaces. Integrating traditional ERD/ERS, source and decoding analysis provide a more comprehensive developmental description of the neural basis of such behaviors.
This setup allowed us to exploit the advantages of the MoBI approach 2,3 and study the social interactions between the infant and the experimenter as they naturally occur without restrictions.
The protocol, from the time the subject arrives to the time he/she leaves, takes approximately 1 hr to complete. Database usa The IMU/EEG setup time and electrode location acquisition varies from 15 – 25 min depending on factors such as hair length and cooperation of the subject. Database url The initialization and configuration of the equipment adds up to 10 min, and the testing session lasts approximately 15 min. Database uml Removal of the IMUs and EEG cap, including cleaning the head of the infant from the hypoallergenic gel, takes 5-10 min.
The following protocol was examined and approved by the Institutional Review Board at the University of Houston. Database uses All infant subjects’ parents or guardians received, read, and signed a consent form prior to participation. Database union Parents received free parking and a $20 gift card as compensation for their participation in the study whereas the infants chose an age-appropriate toy.
The recruited infant subjects met the following criteria: 1) Age between 6 and 24 months. Database update 2) Infant was healthy, had normal growth and development, and had no history of natal issues, concussions, seizures, strokes, or learning disabilities. Database utility Examples of some common developmental disabilities that would exclude an infant from testing are failure to thrive, undernourishment, and use of alcohol or drugs by the mother during pregnancy.
A simplified flow diagram for the simultaneous recording of multimodal neural activity via scalp EEG and IMUs in freely behaving infants is presented in Figure 1.
Measure the infant’s head circumference in cm. Database unit testing Place a measuring tape around the widest part of the head, passing it over the eyebrows and around the occipital prominence in the back of the head. Database user interface Note: Measuring the head circumference is necessary for selecting an appropriately sized EEG cap. Database usa reviews There are special sizes for the infant population described in Table 1.
Starting with ground and reference electrodes, use a small syringe to inject electrolyte gel into the space between the scalp and electrode until the impedance of each electrode measures below 60 kΩ. Hollywood u database This is indicated by a yellow or green light on the electrodes. Uottawa database Further details on EEG electrode preparation are available in 4. Yorku database Note: During the gelling procedure, the infant might move his/her head for various reasons (curiosity, fear, diverted attention). Database view It is therefore recommended that the second experimenter or parent continues to distract the infant. Database vs spreadsheet Use the syringes from behind the infant’s head to avoid the risk of hitting the infant’s face with the needle due to unexpected movement of the subject.
Place video camera (18) so that the infant (12), the actor (14), and the LED trigger (13) are all clearly visible. Database version control See Figure 4. Database viewer Note: Video recordings are used to allow visual annotation and segmentation of behaviors triggered by the experimenter, as well as confirmation of the beginning and end of the recording session.
Solve the forward model. Database visualization Note: The forward problem involves the definition of a head model that represents the geometry of the head and of the electrical conductivity properties of the head volume 9,10.
Solve the inverse problem Note: The inverse problem tries to infer a location, strength and a time course of a source in the brain from the scalp EEG signals by using signal processing techniques 11.
Visually analyze the estimated sources and pick up the solutions that most match your expectation based on prior knowledge from the nature of experiment and interpret them carefully. Database vault Note: This step is the most biased one due to the ill-posed nature of the inverse problem, that is, different configurations of the sources may result in the same surface potentials. Database virtualization Therefore, it may be helpful to perform a sanity check to verify that a given head model and inverse method works well.
Perform a sanity check, localizing a known simulated dipole. Database vs server Namely, for the given head model place a dipole with known configuration, and solve the forward problem to obtain simulated voltages for this dipole.
Figure 8 displays sample EEG and acceleration data from the IMUs for a 16 sec time window recorded during interaction between the subject and the experimenter. Database versioning EEG data was re-sampled at 100 Hz and then band-pass filtered [1-40 Hz] using a 3rd order, zero-phase Butterworth filter. Database vendors Channels with high impedance values (Z > 60 kΩ), and peripheral channels, were discarded 12,15. V database in oracle The IMUs recorded nine signals at 128 Hz: magnetic flux, angular velocity, and linear acceleration in the three Cartesian axes. Database website Here we show the magnitude of the gravity-compensated (GC) acceleration. Database wiki The effect of acceleration due to gravity was compensated by applying a Kalman filter to predict the IMU orientation in a global frame 16. Database workbench Data was segmented by visual inspection of the video recordings (Step 7.1). Database website template Vertical solid lines indicate the start of a behavior of interest, as dotted vertical lines represent the ending of the event.
Influence of motion artifacts is present in the EEG data shown in Figure 8 around 709s. Database works The unconstrained approach to data collection in this experimental protocol makes the EEG data susceptible to eye blinks, eye movements, motion and electromyographic artifacts. Database web application The data was preprocessed by using a 3 rd order zero-phase Butterworth band-pass filter to constrain it to the delta-band (1-4Hz), and standardized by subtracting the mean and dividing by the standard deviation. Database wordpress High-amplitude artifacts were removed automatically using the Artifact Subspace Removal (ASR) method 17. Database weak entity Additionally, peripheral channels are excluded from the data analysis in an effort to minimize myoelectric artifact contamination. Database weekly Frontalis and temporalis muscle contractions merge with EEG signal most prominently at peripheral locations: Frontalis contraction shows up in anterior locations, and temporalis contraction shows up in lateral frontal and temporal locations 15.
To inspect the nature of the data collected with this protocol, EEG data histograms were plotted in Figure 9. Database worksheet In Figure 9A, it describes the data distribution of the standardized signal from three spatially representative electrodes. Database xml The EEG data shows a multimodal distribution for the analyzed behaviors. Database xcode In Figure 9B the kurtosis values are presented as bar graphs for easier visual inspection of the data.
Classification was performed by extracting time-based lags of each EEG channel, reducing dimensionality while preserving the local scatter of each class (Local Fisher’s discriminant analysis (LFDA)) 18, and training/testing a model of the reduced set of features (Gaussian Mixture Models (GMMs)) 19. Database xampp Training/testing samples were randomly sampled over 20 iterations ( i.e., cross-validation) to prevent any over-fitting.Training/testing sample size varies given the number of rejected channels ( i.e., impedance greater than 60 kHz), length of experiment session, and number of trials and behaviors expressed. Database xe However, the number of training and testing samples used for each class (behavior) correspond to 50% of the least populated class. Database xls As an example, the testing set size of each class is N = 1,069 samples for the infant data shown in Figure 10. Pokemon x database All pre-processing and classification steps were computed under the MATLAB programming environment.
Figure 11 depicts the whole procedure for EEG source estimation performed in this study in a step-by-step manner. Os x database More details about each step is also summarized in section 8.
Figure 12 shows results of event related (de) synchronization (ERD / ERS) in the mu rhythm (5-9 Hz) and the dipole sources during the “Reach-Offer” task. Database yml ERS and ERD was calculated as the percentage of a decrease or increase in a frequency-band power which occurs during event (reach and offer task) interval as compared to the reference interval (a segment taken prior to the event). Database youtube This figure also shows gravity-compensated magnitude acceleration obtained during the task from both wrists of the infant and the actor. Database yugioh For a dipole analysis the EEG signals were decomposed using independent component analysis (ICA) to eliminate background noise. Database yml mysql The source estimation was performed in mu rhythm after ICA preprocessing via a fixed MUSIC algorithm 5. Pokemon y database As expected, the sources were localized over the right primary motor area while the subject was using his left hand to grasp the object.
The protocol described presents a methodology to collect data from freely-behaving infants while they are interacting with an experimenter in real time. Dayz database It employs mobile brain imaging technology (scalp EEG) to capture neural activity while simultaneously recording kinematic data with IMUs in strategic body locations. Gpu z database The experiment session is also recorded by a video camera. Cpu z database The three data recording systems are synchronized using a custom trigger system.
The EEG and IMU system are strapped to the subject as he/she is freely moving during the experiment session. Dayz database map The IMUs need to be strapped securely to be able to capture kinematics accurately. Z wave database To ensure full unconstrained mobility of the subject, the equipment has to be as minimally intrusive as possible; thus the use of the holder to support the EEG’s electrode cables and the EEG control box. Database 101 The experimenter then interacts with the infant for approximately 15 minutes. Database 1 to many The infant will elicit an age-dependent repertoire of behaviors during the interaction. Database 12c new features These include rest, reach-grasp, reach-offer, explore, observe, and imitate. Database 11g However, some infants will be unwilling to cooperate in the session due to fatigue, lack of comfort, or stress. Database 1 to 1 relationship Make sure to schedule the experiment when the child is most dynamic and active to prevent the occurence of negative responses from him/her.
The nature of the experiment presents risks to the quality of the data recorded throughout the session. Database 2013 Therefore, it is crucial to test all the connections and data quality before starting the recording session, and to monitor them continuously during the session. Database 2016 If the data EEG system is not recording quality data, stop the software and unplug all connections. Database 2015 Before restarting the software or connecting the equipment back to the laptop, remove all possible noise sources ( i.e., power supplies) from the proximity of the recording hardware. Database 2000 The EEG hardware includes signal amplifiers that can pick up environmental noise if placed close to electrical noise sources. Database 2010 For the IMU receiver, make sure there is no interference in the line of sight between the receiver and the experiment and infant.
This experimental setup provides high temporal resolution neural data by measuring electrical activity at the surface of the scalp. Database 3nf Recent studies have demonstrated the feasibility of utilizing these signals, together with whole-body kinematics, to identify classifiable information for expressive movements 20, and functional movements 21,22, suggesting that this proposed data collection approach could lead to a better understanding of the neural basis of imitation in infants.
Recent contributions featuring powerful machine-learning algorithms applied to brain dynamics 13,20,21 are building a growing toolkit to study surface potentials in more natural settings. Database 3nf example This proposed setup provides a spectrum of possibilities for research questions to be addressed 2,22. Database 360 Particularly, it can be applied to research focused on a) understanding the neural basis of cognitive-motor development of infants based on a large population of subjects; b) understanding the neural basis of the baby’s intent in ‘action and context’, which should be predictive of the incoming behavioral action; c) quantifying common and unique neural patterns to characterize individuality and variability in the developing brain; and d) studying the emergence of imitation and learning processes. Database 3 tier architecture These goals entail the deployment of machine learning algorithms that can deal with statistically rich data both in informative brain-originated potentials and movement or muscle artifacts 12,20,23 .
This study attempts to estimate the cortical sources and electric field potentials using the infant EEG data. Database 3 normal forms Due to the technical difficulties such as the lack of knowledge in infant head conductivity values and thickness of cortical matter, the accurate modeling of the head model is a difficult task. Database 3d Further studies are needed for noninvasive regional tissue conductivity estimates in infants 24. 3 database models Cortical surface segmentation of infant MRI data presents an additional challenge due to the poor contrast found in images of the developing human brain 25. Database 4th normal form Future research is needed to address these difficulties and estimate various neurophysiological correlates of infant development and behavior.
Finally, the proposed experimental protocol and methods could be deployed in the study of those with developmental disabilities such as infants with probable autism spectrum disorder (ASD). Database 4d In such an application, it would be desirable to include a control group and suitable developmental assessments to characterize the two groups (control and ASD). Database 4500 For example, a study group could consist of all high-risk (for ASD) infant siblings assessed with the Autism Diagnostic Observation Schedule 26, symptom severity 27 and the Mullen Scales of Early Learning 28 to characterize general cognitive ability. Database 4 net If available, diffusion-weighted MRI scans would also be highly desirable 29.
This work was supported by Eunice Kennedy Shriver National Institutes of Child Health & Human Development (NICHD) Award # P01 HD064653-01. Database 5500 The content is solely the responsibility of the authors and does not necessarily represent the official views of the NICHD or the National Institutes of Health.