Exploration and virtual experimentation in a local neuroscience database

ELSJZVIER

Journal

of Neuroscience

Methods

63 ( 1995) 1.59- 174

Exploration and virtual experimentation in a local neuroscience database Bret E. Peterson * Section of Neurobiology.

Yale Unitiersiry

Received

28 November

School of Medicine, 1994; revised

333 Cedar

Srreer. New Hauen.

CT 065/R

(:.%I

5 July 1995; accepted 27 July 199.5 -

Abstract Work is currently being done by a number of groups to investigate the possibility of creating one or more neuroscience databases. These databases could provide: (1) a means for conveying complete descriptions of experiments; (2) a platform for virtual experiments; (3) and an interface where modeling and experimental resuIts could be exchanged. This paper &scribes work towards creating a local database with these capabilities by creating a set of experimentai neurophysiology sofmare to& that are tied to a database of experiment descriptions via external scripting mechanisms. These tools provide a means of exploring results and data that can lead to insights into the data that suggest new avenues of research. Because the scriptability of the tools aBows for automation of re-analysis of the data, virtual experiments for testing new hypotheses over large datasets becomes possible. Such vimtal experiments are particularly relevant to computational neuroscience, as they allow hypotheses generated by models to be immediately tested without further collection of experimental data. Keywords:

Neuroscience;

Database;

Virtual

experiment;

Exploration;

Software _---.

-.-___-_._

1. Introduction

that demonstrates how some of these features might be implemented in a local database. Future work will address

Many potential uses for a neuroscience database have been considered (Pechura and Martin, 1991; Huerta et al., 1993). Most of the uses that will actually occur are probably not yet imagined but will arise dynamically from the increased availability of information. The insertion of informatics into neuroscience promises not only to aid in the handling of large quantities of data but to change qualitatively the nature of information that is communicated from one scientist to another and to the scientific community at large. To avoid limiting the potential uses of a neuroscience database, one might best state its role as simply to give

how these features can be expanded to perform on a larger scale. There are many ways in which a database could make descriptions of experiments more complete than is cur-

neuroscientists better tools for doing their science. Such a database should include the following: (1) complete descriptions of experiments;

(2) a pfatform for virtual experi-

ments; and (3) a compilation of animal and model results. A database with these characteristics will be invaluable in enhancing the cooperative effort that is almost certainly necessary if the brain is to be understood. This paper describes a set of software tools on the Apple Macintosh

* Corresponding author: Tel.: E-mail: [email protected]. Elsevier Science B.V. SSDf 0165-0270(95)00106-9

(203)

785-5844,

Fax: (203)

785-6990;

rently possible with the scientific paper. Video and other forms of media, for example, couId be used to demonstrate procedures, stimuli, behaviors, and complex results. This paper focuses on a method for making experimental descriptions more complete by creating an environment that allows the user to move from high level results through intermediary results down to the raw data. This

method is implemented by including the tools for data acquisition and analysis with the data. These tools afford the user the opportunity to explore the data. Such exploration can give the user a more intuitive feel for the data

than one obtinins from a highly polished set of results. This intuition can often lead to new directions of research that are along different paths than those of the original researcher’s interests. Once such paths are identified, it might often be possible to test new hypotheses on old data. Though only a small amount of information may be obtainable from any one data set, the ability to accumulate information over a large number of datasets will allow many hypotheses not

160

B.E. Peterson/Journal

of Neuroscience

considered by the original experimenters to be tested. Again, the inclusion of analysis tools that operate on the data with the data itself provides a straightforward mechanism for implementing these virtual experiments. Computational neuroscience is providing a unique mechanism for testing our understanding of the nervous system. Models are derived from animal experimental data. In turn, models can lead to new hypotheses that suggest new animal experiments. A small example of this process is described in the paper as well as how it might lead to a hypothesis that would be testable by a virtual experiment.

Methods

63 (199s)

159-174

The examples described below are based on somatosensory neurophysiology experiments in the hand representation of owl monkeys and the whisker representation (barrel cortex) of rats. Many of the general principles described, however, are relevant to the development of any scientific database and the corresponding tools. Likewise, though all of tools described are Macintosh applications, coming technologies promise to provide platform-independence so that only applied principles will be of issue. Some of the examples are admittedly a bit contrived. There is some inherent difficulty in describing the richness of a non-paper description from the confines of a paper. Even from these

lab.tiff

Fig. 1. Using the MAP program. Before the experiment begins, regions are drawn to define anatomical areas. The regions shown (A) would be labeled ‘mid d3’ and ‘dist d3’ indicating the middle and distal segments of the 3rd finger. Penetration sites are marked on an image of the cortex (B). When a RF is drawn for a penetration site (Cl, the penetration site is classified based on the previously defined anatomical region the RF most overlaps with. In this case, the penetration site is classified as ‘dist d3’. Repeating this process for a large number of penetration sites allows the user to create a map of the sensory representation. A simple mechanism for exploring with MAP involves clicking on penetration sites in the cortical window which results in the RF for that site being displayed in the sensory window (B,C).

B.E. Peterson/Journal

of Neuroscience

small examples, however, it will hopefully be apparent that such tools provide a powerful means for exploring and understanding data and testing it in new ways.

2. Basic tools

This section describes some of the basic software tools that were developed for doing neurophysiological experiments. Because the individual tools are not the focus of this paper, the descriptions are brief, and many features are not mentioned. The tools were designed with intercommunication capabilities that provide a means not only for more automated experiments but also of using the software as a tool for presenting the experiment and its results. Similar to the idea of encapsulation in object-oriented software design, both the data and the software used to collect and analyze it become part of the database. Encapsulation of software and data offers two important functions: the ability to accurately reconstruct an experiment and the ability to further utilize the data by reanalysis. The tools are written for the Apple Macintosh line of computers and represent several years of work on tens of thousands of lines of C code. The current versions run native on 68000 machines with math co-processors. Native Power Mac versions are also being built. The amount of memory required by each program depends on the experiment, but a machine with 16 MB is usually sufficient. The tools are available via the World Wide Web (WWWI and ftp at SenseLab, a Human Brain Project pilot site, at http://paella.med.yale.edu/senselab/bptest/Bre. html#software. The HyperCard stacks can be put together very quickly once one or two prototypes are established. Adding new experiments to a stack requires little if any more work than normal cataloging of experimental data. Example stacks similar to those described in this paper will be made available at the WWW site mentioned above. 2.1. MAP: a cortical mapping tool

Detailed physiological maps of the cortex can be made by making many microelectrode penetrations into the cortex and measuring the receptive field (RF) of the neurons at each site. These high-resolution maps are used to study how cortical representations reflect behavior (e.g., Jenkins et al., 1990; Recanzone et al., 19921, how they vary across species and individuals within species (e.g., Merzenich et al., 1987), how they change over time in adult animals (e.g., Kaas et al., 1983; Merzenich et al., 1987; Pons et al., 19911, and how they are utilized in the working brain (e.g., Nelson and Bower, 1990). In order to study these cortical representations in greater detail, it is necessary to be able to create many detailed maps efficiently and to be able to analyze the data quickly. To aid in this process, a customized program called MAP (Peterson and Merzenich, 1995a) was developed which

Methods

63 I 199.5) /59-

I 74

161

facilitates both the experimental process as weil as the subsequent analysis. A basic experiment using MAP involves loading images of the cortical surface and the corresponding sensory area being mapped. Each sensory image is divided into named regions (Fig. 1A). Penetration sites and sensory RFs are then marked directly on the appropriate images (Fig. lB,Cl. The named regions in the sensory images are used to characterize the RF for each penetration site. MAP provides a simple mechanism for maintaining an accurate record of all penetration sites sampled during the course of an experiment. After completing the experiment, MAP allows for a variety of analyses. These include distance and area measurements, reclassification of penetrations based on multiple criteria, and of course building of maps. 2.2. EXP: an electrophysiology

tool

EXP attempts to incorporate most of the necessary functions for performing a quantitative neurophysiology experiment into a single package (Peterson and Merzenich, 1995b). The program collects analog signals, outputs analog waveforms, and performs some of the early processing, consolidation, and analysis of collected data. The program interacts with MAP so that quantitative data is automatically associated with the penetration site from which it came. This information can later be used to reconstruct the response to a stimulus at many penetration sites across the mapped cortical zone. EXP contains a number of features that allow for automation of data acquisition and analysis. For example, a set of histograms can be defined for a set of presented stimuli which are then generated for all penetration (recording) sites in an experiment. This automation helps relieve some of the tedium of testing new ideas on massive quantities of data. Because EXP is part of the database as are the files describing the acquisition protocols, EXP can be used to directly view the protocol by displaying the stimuli presented as well as the parameters controlling the acquisition. It can also be used to observe other relevant data often not shown in scientific papers such as calibration data. In our threshold experiments on rat barrel cortex, we include recordings that show the actual stimuli (recorded from a linear variable differential transformer (LVDTI) rather than the voltage signal applied to the stimulator. There is some overshoot seen in the signal, and though it does not appear relevant for our purposes, it does provide a more accurate description of the experiment that might be useful to another researcher looking for subtleties in the responses at some future time. 2.3. CAMERA: an intrinsic imaging tool

CAMERA shares many features of both MAP and EXP. Like MAP, it is able to grab frames from a charge-coupled

B.E. Peterson/Journal

162

of Neuroscience

skin

74

2.4. NET: a self-organizing

network

modeling tool

NET is a program which all:jws development and analysis of Kohonen-like networks. These networks have been used to model self-organization of representations as well as plasticity effects that have been observed in adult mammalian cortex (Grajski and Merzenich, 1990a, b). The networks consist of layers of units whose outputs reflect the sum of their weighted inputs passed through a threshold function. A Hebbian-based rule is used to increase the strength of connections between units with correlated outputs (firing rates). NET allows the user to create 2-dimensional layers of cells. All cells within a layer share the same connectivity pattern. After all of the layers have been defined, the connectivity pattern from each layer to all other layers is

skin

oortexl

cortex-l

63 (I 9951159-l

The trade-off, however, is that many images must be averaged before a significant signal can be obtained. CAMERA contains a number of experimental protocols, all which revolve around the averaging of frames grabbed from a CCD camera. Using these protocols, CAMERA has been able to obtain signals from images of the cortex that reflect the underlying neurophysiological activity (Peterson and Goldreich, 1994).

device (CCD) camera. Because this function is the focus of CAMERA, it is of course able to do much more with the frames once they have been grabbed. Like EXP, CAMERA can also control D/A channels on a Lab-NB board (National Instruments). The control of these channels is not as sophisticated as it is in EXP, but it is enough so that different stimuli can be presented which are synchronized with the capturing of frames from the camera. This program was written to facilitate a technique which uses intrinsic changes in the optical properties of the cortex to measure correlated changes in neurophysiological activity (Grinvald et al., 1986). These signals manifest themselves as changes in the absorption of specific colors of light. By recording these changes with a camera, it is possible to indirectly view cortical activity. One difficulty with this technique is that the signals are quite small. The change in reflectance can be less than 0.1% of the reflected light (Grinvald et al., 1988). Measuring this change with g-bit images (a resolution of N 0.4%) was accomplished by averaging frames in software. By using modest hardware to make these measurements, which is more than an order of magnitude cheaper than the equipment originally used for this technique, the technique is made more accessible and therefore more useful to a large number of laboratories.

skin

Methods

cortex1

I

cortexl

cortex-l

Fig. 2. The NET program allows the user to define layers of units that can be connected together. The input pattern to a unit can be shown by clicking on it with the mouse (A). Here the network is untrained so all weights are of equal values (gray). The selected unit in the layer labeled ‘cortex-E’ receives input from a 5 X 5 region in the ‘skin’ layer and a 5 X 5 region in the ‘cortex-I’ region. The skin layer represents the sensory input layer that projects to the excitatory cortex-E layer. The cortex-E layer projects to the inhibitory cortex-1 layer which in turn projects back to the cortex-E layer. Using a trained network, a 3 X 3 stimulus is placed on the skin layer. A response is elicited in the cortex-E layer (B). As the response in the cortex-E layer increases, it begins to excite the cortex-1 layer (0. As the cortex-1 response grows, it inhibits the response in the cortex-E layer CD).

B.E. Peterson/Journal

of Neuroscience

defined. Along with the size and shape of connectivity patterns, the user also defines which connections are plastic (subject to the Hebbian learning rule). The cells used in the models shown here represent many neurons. In the cortical layers, each cell represents approximately a 100 X 100 pm column, whereas on the skin, each cell represents approximately a 500 X 500 pm patch of skin. Once the network has been defined, the user can either run it by hand and apply stimuli by clicking with the mouse, or they can create a set of stimuli with the stimulus editor that can be presented any number of times. The stimulus editor allows any spatiotemporal pattern to be defined. It also allows the locations and relative frequency for each location of stimulus presentations to be defined. Once the stimuli are defined, they can be applied to train the network. NET displays the network of layers as a set of 2-dimensional grids (Fig. 2). The user can easily view input weights, output weights, activation levels, or output levels. They can also select the activation or output levels for any cell(s) to be plotted over time. Receptive and projective fields can be generated and displayed in a similar fashion. 2.5. HyperCard

Methods

63 (19951 159. !74

!h7

that move the user from one card to another can be made in the stack so that the user can be guided through the data in a logical fashion. HyperCard has been used as a front end for larger database packages and thus provides a reasonable mechanism for expansion when the size of the data overwhelms its database capabilities. HyperCard supports two languages: HyperTalk and Applescript. HyperTalk was the original HyperCard language that was used to program responses to user actions on buttons and fields (text). AppleScript is a newer language that is similar to HyperTalk but also provides mechanisms for controlling other software applications that have been made ‘scriptable’ by their authors. In the past, applications often included their own scripting or macro-language, but AppleScript is much more powerful because a single script can access multiple applications. A script might use information from one application to determine how data in another application is processed. AppleScript also provides consistency so that one does not have to learn a different language for each application. The ability to control a number of programs from a single program via scripts provides a powerful mechanism for simplifying data access and for providing a higher level of task automation.

and AppleScript

HyperCard (Apple Computer) provides a data-storing and data-presenting environment that is loosely based on a stack of index cards. Information for an individual entry is stored on a card. A series of cards make up a stack. Links

3. Exploration

MAP, EXP, and CAMERA can be used to review data collected during an experiment. Exploring the data with

Fig. 3. Cortical magnification can be measured by defining a small region of skin (A) and then reclassifying the penetration sites to see which has a RF that overlaps that site (B). Only 3 penetration sites shown as triangles (80, 94, and 30 - numbers situated to the upper right of the penetration site) had RFs that overlapped this site. The sites are not continuous and have other penetration sites between them (shown with dark line segments).

164

B.E. Peterson/Journal

of Neuroscience

these tools can lead to recognition of patterns in the data that might otherwise be missed. Though each of the individual tools contains mechanisms that aid in exploration, exploration becomes truly powerful if a database is used to tie related information together. Such a structure allows the user to quickly move amongst related data. By including scripting in the tools, data access becomes transparent, and the user can concentrate on the logical structure of the data (related experiments) rather than its physical structure (where individual files are stored and what formats are used). This section describes how the tools can be used for exploration, and how this process might lead to new questions about the data. 3.1. Simple exploration with MAP The MAP program can be used to look at the RFs associated with each penetration site. When a penetration site is clicked on, any RF that was drawn for that sight is shown in a sensory window (Fig. lB,C). By repeating this procedure, a user can look for patterns in how RFs are laid out in the cortical representation. Cortical maps are generated based on the RF at each penetration site. Another point of interest that can be inferred from these measurements is cortical magnification. Cortical magnification is the inverse of a RF in that it refers to the area of cortex that is excited by a single point on the sensory surface. To measure this variable with MAP, a small region can be created on the sensory surface, and then the classification of all of the penetration sites can be recalculated. Only those penetration sites that have a RF that overlaps the defined region will be classified. In essence, the classified penetration sites will comprise the cortical representation for that sensory site. Its area would be a measure of cortical magnification. An example of this process is shown in Figs. 3 and 4. In this example, only 3 sites were found that had RFs overlapping the defined region on the distal segment of the middle finger. Surprisingly, these sites are not continuous. By clicking on penetration sites, we can then look to see how the RFs of the classified penetration sites and their neighbors overlap the specified point of skin (Fig. 4). The RFs do seem to move in a continuous manner across the skin. With further exploration we might address an interesting question that this data suggests: at a finer scale, are there disruptions in the topography or does cortical magnification require a denser sampling to infer it? The example suggests that at least the latter is true, but the former is also likely to be true and would be important to modellers of self-organization.

Methods

63 (I 995) 159-l

74

analysis features revolve around the histogram which are accumulations of spikes (action potentials). Many features of the histogram are definable such as which trials are used, what time window is of interest, and should all spikes be included or just the first spike in the time window for any one trial. EXP also generates statistics for the histograms such as mode, mean, and standard deviation. In a threshold experiment, we are often primarily interested in the total spike count produced by a stimulus as compared to the spontaneous condition. Upon looking at the individual histograms (Fig. 5A), however, we might notice that there also seems to be a time shift associated with the increased response. To study this phenomenon in more detail, we redefine the histograms in EXP to look at the first spike that occurs in the first 25 ms after the stimulus is presented (Fig. 5B). The shorter response time for the larger stimuli is clear. EXP could then be used to

penetration

30

penetration

63

penetration

94

penetration

93

penetration

80

3.2. Simple exploration with EXP EXP was designed to automate both data acquisition and analysis. It thus has a large number of capabilities that allow for reanalysis of large quantities of data. The main

Fig. 4. By clicking on each penetration site in Fig. 3, their RFs can be comparedto each other and to the defined skin region. Exploration in this manner can raise interesting questions abut the topography as well as develop insight into the limits of resolution of the mapping technique.

B.E. Peterson/Journal

of Neuroscience

measure this latency shift in even more detail. This large latency shift poses a number of questions to its origin which could then be addressed by further experiments and/or realistic physiological models. 3.3. Simple exploration with CAMERA

Viewing intrinsic images on paper requires practice in trying to correlate corresponding areas. CAMERA contains features that allow images to be shown as continuous or stop-frame movies. Switching between two images that are already aligned greatly improves the speed at which one can recognize changes in the images. By collecting images to responses of individual whiskers, a map of barrel cortex can be made (Peterson and Goldreich, 1994). By showing these images in sequence, we obtain a feel for the layout of the map as well as the degree of overlap in the images. The large degree of overlap in the images that is observed for large stimuli raises questions of whether these are an artifact of the intrinsic signal or a true reflection of the underlying physiology. Another means of exploration with CAMERA is by creating new images. The raw data is stored as average images that consist of images that were averaged over many frame captures. Difference images are created by

A

Methods

63 (I 9951 IS9

I74

16~5

subtracting an average image that was collected before stimulation from an average image that was collected during stimulation. But any two average images can be subtracted. If an experiment using multiple stimuli was done using this technique, we might want to subtract two average images collected during different stimuli rather than using the pre-stimulus average image. We are just beginning to explore some of the image data in this way. 3.4. Integrated and CAMERA

exploration

with HyperCard,

MAP, EXP,

A prototype database was created that describes a series of intrinsic imaging experiments within a HyperCard stack. The prototype stack is made up of cards that describe individual experiments (Fig. 6A). The stack gives basic database capabilities such as sorting and searching by different fields. For example, the user can click on the Sort. . . button to specify that the stack should be sorted by the date field or by the title field. The Find. . button allows the user to specify a certain string that should be searched for in some specified field. Jn the case shown, the card was found by searching for ‘ 12/ 18’ in the date field. The arrow buttons in the lower right portion of the card allow the user to move through the stack sequentially. A

0.

40

01

Fig. 5. EXP can display histograms as they are gathered as well as for subsequent analysis. A set of histograms displayed during an experiment shows the response to a stimulus series of increasing amplitude (A). Careful observation indicates that there is a latency shift in the response as the response increases. Resetting the time window of the histograms allows further exploration of this shift in the onset response (B).

166

B.E. Peferson/Jolournal

of Neurosc rience Methods

ment. In the card shown, there is data for the MAP, EXP, and CAMERA programs, but not for the Canvas program. By selecting the program and clicking on the View: button, the user is taken to a specific software card (Fig. 7A) for the experiment. In this case, the card shows the data that was collected with the MAP program. There is only one file called ‘map data.12/ 18’. By clicking on the Load File: button, the MAP program is invoked and

specialized card exists that allows all of the experiments in the stack to be listed. Filters can be selected so that the list only contains experiments with certain attributes. Each experiment card has a Data button that takes the user to a card describing the data that has been collected for that experiment (Fig. 6B). The stack has a modifiable list of software applications, and this card indicates which applications have data associated with them for this experi-

Threshold

Imaging

Experimenters: Bret Peterson Dan Goldreich

General

63 11995) 159-174

Protocols:

(Check)

StandardElectraphysiology Oatiel ImaW@

I

Description:

Looked TVseehog thresholdsmeasuredusing;the

optical imaging technique compmd do those u&g stendenl elecWoph@ology Wmiqu~~s. Used this experiment for figures in thesis.

Experiment18of26

Threshold fsnaging

MAP Data - 1 (I[ 'rogram

Data:

MAP Data EXP Da& CAMERA Da&

No Program

CanuasData v Data:

CenvesData

Fig. 6. Using a database to explore an experiment (see text). Cards in the database describe experiments (A) and help locate results and data (B, and Figs. 7AB, 8A,B, 10, and 11). The user also has access to driginal data such as the average images used to create the difference images (Fig. 8C). The average image contains landmarks that allow the difference image to be compared to the cortical map (Fig. 9A-C). The estimated representation of the Dl whisker from the map (drawn in Fig. 9A,B) differs somewhat from the ellipse fitted to the image by the CAMERA program (Fig. 9C). Penetration site 11 is much further from the border of the darkened area of the image than is penetration site 10. Using the database to access the quantitative neurophysiological data (Figs. 10 and 1l), the responses for the three marked sites to a Dl whisker stimulus are obtained. Though the response at penetration site 10 is smaller than those with the Dl representation (penetration site 5). it is stronger than that at more distant sites (penetration site 11).

B.E. Pererson/Journal

of Neuroscience

automatically loads the specified file (Fig. 7B). The file contains a map of a single whisker representation (Dl) in the barrel cortex of a rat (dark gray circles indicate penetration sites that had a primary response to the Dl whisker). Threshold data was obtained for each penetration site with the EXP program. Returning to the data card, the user can select to view data collected by the CAMERA program. The CAMERA card (Fig. 8A) shows that 4 files were collected; each file corresponds to an intrinsic optical image created using near-threshold stimuli of increasing magnitude. Clicking on the Load File : button for the image using the largest stimulus takes the user to the CAMERA program displaying the selected image (Fig. 8B). Continuing in this manner, the user can view the rest of the data collected during the experiment. The buttons in the lower left take advantage of CAMERA’s scriptability and allow the user to

Threshold

Methods

63 (1995)

IS9 - I74

control how the image is displayed without having to switch control to the CAMERA program. The files listed in the database present data that reflect the purpose of the experiment, namely comparing threshold physiology and imaging data. However, they do not reflect all of the information collected during the experiment. Each image, for example, is created by subtracting two images. An image collected prior to the stimulus is subtracted from an image collected during the stimulus so that the final image shows the change in absorption. The final images tend to have few landmarks as they are subtracted from each other in the two images, Because the data is presented with the software used to collect it, it is easy to access the original files from which the processed data was created. The user can open one of the two average images used to create the difference image (Fig. 8C). On this image, blood vessels serving as landmarks are

12118192

Imaging

t-113-al Description: Map of Dl bam?l.Penemions 17,18, and 19 me& lesion sites.

161

Extended Path: Vmsholii:12118192:

File Name(s): nmp datR.12118

Fig. 7. Seelegendof Fig. 6.

$1

B.E. Peterson/ Journal of Neuroscience Methods 63 (1995) 159-l 74

168

. ..

CAMERA

Data

Threshold Imagisg

12t1w92

General Description: Thresholdimgesfor0.4Q,0.2Q,0.1Q,and 0.05Q.

Extended Path: theshold:12t18t92:heges:

File Name(s): dl.O.rlQ.avg

Fig. 8. See legend of Fig. 6.

Fig. 9. See legend of Fig. 6.

$1

B.E. Peterson/Journal

ofNeuroscience

Methods

63 (1995)

159.-174

169

EXP Data Threshold

Imaging

12/18192

GeneralDescription: OnQ a few pemtmion3 were made.

Extended Path: th_resho~:l2118/92:quande.ta:

File Nm(s):

$1

’

I

testlo thresh.exp

/-FGZSGl pen *:I

seq. str: 71

Fig. 10. See legend of Fig. 6.

much clearer. The user can now compare the difference image to the physiological map (Fig. 9A) directly by overlaying the blood vesselpattern and penetration sites on the average image (Fig. 9B) and then loading the difference image which is displayed beneath thesemarkers (Fig.

9C). The user can verify that the location of the penetration sites responding to whisker Dl (1,2,3,5,6,7, and 8) correspondclosely with the image produced by stimulating whisker D 1, Fig. 9C shows the ellipse fitted to the image by the

Fig. 11. See legend of Fig. 6.

170

BE.

Peterson

/ Journai

of Neuroscience

CAMERA program whereas the hand&awn estimate of the Dl whisker representation based on the physiological map is shown in Fig. 9A,B. From the image, it appears that penetration 10 (characterized as responding best to the delta whisker) is much closer to the delta/D1 border than is penetration 11 (also in the delta representation). To test this idea further, the. user can return to the data card and go to the card for EXP data (Fig. 101. Clicking on the Load File : button initiates the EXP program and loads the file with the threshold histograms. The lower left of the card contains a button and field that allow the user to load spikes for individual penetration sites. Again the scriptability of the tools makes data access convenient by allowing control of other applications without leaving the Hyper-

A

Penetration

Methods

63 (I 9951159-i

Card environment. Spikes are loaded for the DI stimulus for penetrations 5, 10, and 11 (Fig. 11). The response at penetration site 10 is not as strong as that within the Dl representation (penetration site 51, but it is noticeably stronger than the response further from the border at penetration site 11. Again exploration of this type leads to new questions: are neurons within the delta whisker representation but near the Dl representation responding to the Dl whisker stimulus or is this an artifact of the long seeing distance of extracellular recording? The former would be of considerable interest and merits further study. This simple example demonstrates the power of maintaining the link between the software and data. It provides a straightforward method for a normal user to view the

sites ’

I

Run in order Summed histl (paste to Cricket)

[Opsn] /-Eisq

300

74

Ttreshold Response of Dl Whisker

E ‘B B d

200

i

-

0.07*

-

0.14” 0.28”

il C

u 100 E B B 0

0

10

Time (msec)

20

30

Fig. 12. Because MAP and EXP are scriptable, data can be analyzed from HyperCard by collecting it from one program based on information from the other program (see text). The card (A) queries MAP to find all penetration sites that were classified as being in the Dl whisker representation. It then averages the histograms from EXP for all of these penetration sites for some specified stimulus. The average histogram is pasted into a graphing program. Repeating this process for each stimulus produces a graph of the average response to a series of stimuli in the Dl whisker representation (B). This basic script could be more fully automated by making simple changes that allow it to run over all stimuli and over multiple experiments. Once the script is defined, it could be used to run virtual experiments that test new ideas by changing the type of histogram that is calculated by EXP.

B.E. Peterson/Journal

ofNeuroscience

results of an experiment. It also provides a means for a more knowledgeable user to search the data in greater detail and even to seek information not purposely collected by the experiment.

4. Virtual

experiments

Virtual experiments are experiments that test new hypotheses on data that was collected to test other hypotheses. The qualifier ‘virtual’ is used to refer to the data acquisition process where relevant data is collected from existing data sets into a unique data set rather than from a new set of animals. Softky and Koch (1993) recently demonstrated this approach by studying encoding of vari-

skin

Methods

63 C/9951159-

174

171

ability in spike trains by analyzing data that was originally collected to specifically study the various cortices they were from. This section describes how scriptable software tools can be of great value in performing virtual experiments. It then gives an example of a virtual experiment that might be performed if a more complete database were available. This example stresses the need for such databases, shows how they do not necessarily threaten an experimenter’s ‘domain’, and demonstrates how virtual experiments might allow us to test ideas that otherwise could not be tested. 4.1. Using scripts for analysis with MAP and EXP

Up to this point, the scriptability of the tools has been used to bring relevant data to the foreground. Though this

suboortexl

subcortex-l

skin

subo0rtex-E

subcortex-l

skin

subcortex-l

subcortex-E

subcortex-l

Fig. 13. Weights and RFs in a trained model. The model was trained with a 3 X 3 stimulus that was always on one side or other of the finger border (shown as vertical white lines in each of the layers). Input weights are fairly symmetric for sites representing the middle of a finger (A). As the finger border is approached (B and C), they become less symmetric as weights on the other side of the border are close to zero (black). RFs are calculated by stimulating each spot on the skin and measuring the evoked response (D-F). In this simple model, they very closely resemble the pattern seen in the input weights. One noticeable feature is that the RFs near the border (F) are smaller and stronger than those further away CD).

172

B.E. Peterson

/Journal

of Neuroscience

is an extremely useful feature, it only begins to tap the potential of having a scripting language. Scripts can be used to gather information from one program, and then based on that information, programmatically control analysis in another program. An example of this process is shown in Fig. 12. The script has been divided amongst buttons for explanatory purposes, but it obviously could run as a single script. Assume that the latency shift we discovered earlier while exploring with EXP has piqued our interest, and we now wish to see if this phenomenon holds up beyond the single penetration site that we previously studied. To do this, we open a HyperCard stack that contains a card for accumulating results (Fig. 12A). The Open Map button starts the MAP program and loads the cortical map. The List Pens button gets all of the penetration sites and their classifications from the MAP program. The Get Pens button sifts through the penetration sites and collects the numbers of all of those that were classified as being a part of the Dl whisker representation. The Open EXP button starts the EXP program and loads the histograms for looking at the onset response. Finally, the Build Histo button creates an average histogram for all of the penetration sites that were gathered by the Get Pens button. The histogram can either be saved to a file or directly pasted to a graphing program. By repeating this process for all 4 stimulus levels, we obtain an average response for the entire cortical representation of the Dl whisker (Fig. 12B). The latency shift is indeed a robust phenomenon within this animal. As mentioned earlier, for less demonstrative purposes, this entire process could be easily automated and could be extended to include many experiments. Once a script is written, it too becomes part of the experimental description. It is of course straightforward for the HyperCard description of the experiment to contain links to cards containing scripts that automate parts of the analysis. Analysis scripts facilitate virtual experiments because often only small changes are required to automate analysis of some new aspect of the experiment. For example, the script described above could be used to accumulate any defined histograms over a series of penetration sites. By loading a different EXP file, one might look at a different time segment to study the average offset response or look for periodicity in the response by building the average autocorrelation histogram. The goal is for the experimenter to do work to answer a question that resembles the question itself and not to be overwhelmed with the logistics of handling the data. 4.2. A Hypothetical

uirtual experiment

Fig. 13A,B show the input weights and RFs for a simple model that was trained with a 3 X 3 stimulus pattern which did not cross the midline of the sensory surface. This model is intended to represent two fingers which end up having non-overlapping representations in

Methods

63 (199.5) 159-i

74

cortex because their inputs are uncorrelated. The input weights near the finger border are much larger (whiter) on one side of the midline or the other. A similar effect is seen in the calculated RFs. Another effect that is seen in the RFs is that they appear smaller and stronger as they near the border. Presumably, this effect is due to the competition that is enacted in the network by use of a normalization rule for the weights. Inputs are uncorrelated on either side of the midline, so units on one side do not compete well for ‘weight resources’ against units on the other side of the midline. Once a unit is clearly receiving the majority of its input from one side of the midline or the other, a positive feedback loop quickly builds up those weights and diminishes the uncorrelated inputs on the other side. It has been suggested that real neurons also compete in this way, and the question then arises do these stronger, smaller RFs appear at real cortical representation boundaries? The difficulty of studying this hypothesis in real cortex is finding appropriate boundaries. Unlike with the model, it is difficult to ensure that all areas of the fingers are being stimulated equally which is important when assuming a Hebb-based rule with competition. The finger borders are problematic because it is unlikely that the sides of the fingers are stimulated to the same degree as areas near the middle of the finger. Fortunately, the glaborous hand representation usually contains islands of representation of its dorsal side that can appear anywhere within the representation. An experiment could be designed to find cortical maps that have dorsal island representations that interrupt part of a glaborous finger representation and to compare sites near this boundary to corresponding sites on the other side of the midline of the finger which are not on a boundary (Fig. 14). This virtual experiment could be performed today with the tools described above if a database of normal maps existed, but it does not (yet). This hypothetical experiment has two important aspects that emphasize the need to provide a platform for virtual experiments: (1) it would be performed on normal data collected over a large series of experiments that would otherwise disregard this information and (2) it would be the only practical way that this particular experiment could be performed. The data shown in Fig. 1 is from a cortical lesion study. A map was created of the normal cortical representation before inducing a lesion of one of the finger representations. After months of retraining the finger, the region was mapped again to see how the finger representation reemerged (Xerri et al., 1991). The normal map is used only as a reference from which to measure changes in the altered map. Many of the details of the normal representation are unimportant to the purpose of this experiment and others like it, and the finer details might be difficult to generalize over the few examples from any one experiment. However, many interesting virtual experiments could be performed on an accumulation of normal representa-

5.E. Left

Hand

Peterson/Journal

Right (area

of Neuroscience

Cortex 3b) d2 d

d3

Fig. 14. A hypotheticalexperimentlooking for the border effect seen in the modelin Fig. 13. The2nd. 3rd. and 4th fingers are labeledd2, d3, and d4, and their segments are labeledp (proximal), m (middle), andd (distal) (A). The 3rd digit representation is mapped in primary somatosensory cortex (B). The map containsan island of dorsal handrepresentationthat borders the representation of the distal segment of the 3rd finger. Two penetrationsites that have RFs at comparablesites on opposite sides of the finger midlineare marked as circles. Theexperimenttests whether the RF of the penetrationsite near the dorsum border has a RF (D) similar to that (C) of the penetrationsite that is not on a representationborder, or is the RF stronger and smaller (E) as predictedby the model.

tions that were used as controls in many different experiments. If these virtual experiments are not performed, this information is needlessly wasted. The experiment described here requires that the island of dorsal representation be located in a fairly specific position relative to a finger representation, but the location of these islands is highly variable from animal to animal. It would be ludicrous and unconscionable to design an animal experiment that specifically tested this hypothesis because often the dorsal island representations would not be suitably located, and no data would be obtained. By searching through a database of normal representations, however, this experiment could be performed without sacrificing any additional animals. Until such a database exists, this experiment can not be done.

Methods

63 (19951

IS%

174

: 73

Exploration is an important means of getting one’s hands on the data. It is useful for both the original experimenter and anyone else interested in the purported results and their implications. By providing tools for exploration with the data itself, userscan develop an intuitive feel for the data and its nuances that might otherwise be lost in the final presentation of the results. A databaseof experiments that makes use of scriptable software tools as the one described here can help isolate the user from the tedium of locating data and deciphering its format and can aid the user in exploring a variety of data over a range of different experiments. Insights obtained at this level can lead the user to new avenues of research. One method of pursuing new avenuesof researchis by the virtual experiment. These experiments allow new hypotheses to be tested without further collection of data. They can be fairly easily implemented if appropriate tools for handling the data are made available with the data itself. If the tools are scriptable, automating new analyses over large datasets that might even be from different experimenters becomes plausible. Virtual experiments require that open databasesbecome available. They can be used to study information that would otherwise be discarded, and because they could potentiaily be applied to very large datasets,they could addressquestionsthat could not otherwise be answered. Clearly, the next step must involve creating large databasesof experiments that can be sharedacrosslaboratories. Though the examples above are basedon Macintosh applications, the general techniques described promise to be better implemented by using component-basedsoftware technologies that are just starting to appear. These technologies should complement some of the object-oriented databasetechnologies, so that software components can be stored with data and can operate on the data across platforms. The components should provide capabilities similar to those described above so that exploration of data and virtual experiments become commonplace We are currently investigating the use of hybrid technologies such as Illustra that promise to combine high-performance aspects of large databasepackageswith support for complex data structures as those described here. Such systemswill open up neuroscience to a wider range of scientific talent and will inevitably speedits progress.

5. Conclusion

Acknowledgements

Heinz Pagels (1988) wrote “God is in the details of existence. And anyone who refuses to look there is likely to be worshiping (sic> idols ‘I. Despite his being a theoretician, Pagels was trying to emphasizethat we must have a ‘hands-on’ approach to science if we hope to truly understand anything. We must be familiar with the low level data before we can understand the grander theories it might support.

At UCSF, I would like to thank Michael Merzenich for being my mentor in the somatosensorysystem as well as for his continual support during much of the development of the software tools. I would also like to thank Christian Xerri from whose experiment the monkey data presented here was taken, as well as Daniel Goldreich who was my collaborator during the intrinsic imaging experiments. At Los Alamos, I developed the HyperCard database and

174

B.E. Peterson/Journal

of Neuroscience

added scriptability to the software tools. I would like to thank John George and Chris Wood for their support and comments on this work. This work was supported in part by NIH Grants NS-10414 and 2T32GM07449 and by US Department of Energy and Los Alamos National Laboratory. References Jenkins, W.M., Merzenich, M.M., Ochs, M.T., Allard, T., and Guic-Robles, E. (1990) Functional reorganization of primary somatosensory cortex in adult owl monkeys after behaviorally controlled tactile stimulation J. Neurophysiol., 63: 82-104. Grajski, K.A., and Merzenich, M.M. (199Oa) Hebb-type dynamics is sufficient to account for the inverse magnification rule in cortical somatotopy. Neural Computat., 2: 74. Grajski, K.A., and Merzenich, M.M. (1990b) Neuronal network simulation of somatosensory representational plasticity. In: D.S. Touretzky (Ed.), Neural Information Processing Systems, Vol. 2, Morgan Kaufman, San Mateo, CA. Grinvald, A., Lieke, E., Frostig, R.D., Gilbert, C.D., and Wiesel, T.N. (1986) Functional architecture of cortex revealed by optical imaging of intrinsic signals. Nature, 324: 361-364. Grinvald, A., Frostig, R.D., Lieke, E., and Hildesheim, R. (1988) Optical imaging of neuronal activity. Physiol. Rev., 68: 12851365. Huerta, M.F., Koslow, S.H., and Leshner, AI. (1993) The Human Brain Project: an international resource. TINS, 16: 436-438. Kaas, J.H., Merzenich, M.M., Killackey, H.P., and Van der Loos, E. (1983) The reorganization of somatosensory cortex following peripheral nerve damage in adult developing mammals, Ann. Rev. Neurosci. 6: 325-356.

Methods

63 (1995)

159-174

Merzenich, M.M., Nelson, R.J., Kaas, J.H., Snyker, M.P., Jenkins, W.M., Zook, J.M., Cynader, MS., and Schoppmann, A. (1987) Variability in hand surface representations in areas 3b and 1 in adult owl and squirrel monkeys J. Comp. Neural., 258: 281-296. Nelson, M.E., and Bower, J.M. (19901 Brain maps and parallel computers. TINS, 13: 403-408. Pagels, H.R. (1988) The Dreams of Reason: the Computer and the Rise of the Sciences of Complexity, Simon and Schuster, New York. Pechura, CM., and Martin, J.B. (1991) Mapping the Brain and Its Functions: Integrating Enabling Technologies into Neuroscience Research, National Academy Press. Peterson, B.E., and Goldreich, D. (1994) A new approach to optical imaging applied to rat barrel cortex. J. Neurosci. Methods, 54: 39-47. Peterson, B.E., and Merzenich, M.M. (1995a) MAP: a Macintosh program for the generation and analysis of categorical maps applied to cortical mapping. J. Neurosci. Methods, 57 (1995) 133-144. Peterson, B.E., Merzenich, M.M. (1995b) EXP: a Macintosh program for automating neurophysiological experiments. J. Neurosci. Methods, 57 (1995) 121-131. Pons, T.P., Garraghty, P.E., Ommaya, A.K., Kaas, J.H., Taub, E., and Mishkin, M. (1991) Massive cortical reorganization after sensory deaffertation in adult macaques. Science, 252: 1857- 1860. Recanzone, G., Merzenich, M., Jenkins, W., Grajski, K., and Dime, H. (1992) Topographic reorganization of the hand representation in cortical area 3b of owl monkeys trained in a frequency-discrimination task. J. Neurophysiol., 67: 1031-1056. Softky, W.R., and Koch, C. (1993) The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs. J. Neurosci., 13: 334-350. Xerri, C., Jenkins, W., Merzenich, M.M., Peterson, B.E., and Beitel, R. (1991) Reorganization of primary somatosensory cortex and functional recovery from stroke in adult owl monkeys. Sot. Neurosci. Abst.. 17: 843.