Intravenous drug self-administration behavioral experiments on rhesus monkeys and rats conducted at the Preclinical Medical Research Laboratories (1966-1996), which was established by Dr. Tomoji Yanagita with the primary goal of conducting drug dependencee study (photographed by author). Regarding the legitimacy of conducting such experiments and the ethics of animal experimentation, please refer to "4. Ethics of Animal Experimentation" on Operant Behavior in Neuroscience and "7. The harsh reality of drug dependence and drug abuse society" on Drug Dependence Concept.

A Brief Note

In basic research on drug dependence, reproducing excessive drug-seeking behavior in experimental animals is a key challenge. This is because such behavior represents the behavioral manifestation of psychic dependence—a core concept of drug dependence. Excessive drug-seeking behavior in experimental animals can be reproduced and observed through intravenous (or intragastric) drug self-administration paradigms. Given the various behavioral facets involved in excessive drug-seeking, such behavioral analysis is indispensable to research on drug dependence.

Contents of this page

1. Behavioral analysis for drug dependence studies

In the Drug Dependence page of this website, we stated that the essence of drug dependence is psychic dependence, which in humans is characterized by an intense craving for certain drugs. This craving causes compulsive drug-seeking behavior. On the other hand, drug-seeking behavior in laboratory animals can be investigated by drug self-administration behavior. This has been carried out mainly using monkeys and rodents as operant behavior using drugs as reinforcing stimuli. When these laboratory animals respond by pressing a lever, the drug is injected into the animal's body through a catheter that has been implanted in the vein beforehand. When the drug is injected into the vein, it immediately reaches the brain and the drug effect is quickly expressed. This allows the presence or absence of the reinforcing effect of the drug to be detected very sensitively and accurately by the increase or decrease in the lever-pressing response.

When the drug is not soluble in water, there is a self-administration method in which the drug suspended in a vehicle (solvent) is injected into the stomach through a catheter. In this case, it takes a little longer for the drug to be absorbed through the digestive system, reach the brain, and exert its effects there than with intravenous self-administration. However, since humans can develop psychic dependence on drugs taken orally, the intragastric drug self-administration method in animals is also a useful method for investigating psychic dependence. On the other hand, alcohol is water-soluble and is a dependence producing substance that is familiar to humans. However, attempts to get experimental animals to drink alcohol orally usually fail due to the taste and its irritating properties. Intravenous administration of alcohol irritates the blood vessels, making it impossible to induce self-administration behavior in animals. Therefore, in the case of alcohol, a catheter is placed in the stomach of the experimental animal and the self-administration behavior of the alcohol absorbed from the stomac is observed. For the above reasons, intravenous or intragastric drug self-administration behavior experiments are one of the central technics in drug dependence studies using experimental animals.

For information on the operant behavior, intravenous drug self-administration behavior in rats and rhesus monkeys, and smoking self-administration behavior in rhesus monkeys, please refer to this website Operant Behavior in Neuroscience.

In drug dependence studies, psychic dependence and the resulting drug-seeking behavior are the main themes. However, various behavioral factors are involved in various aspects of drug dependence. In that sense, understanding various behaviors in addition to the drug-seeking behavior is also an important clue to understanding drug dependence. Here, we have attempted to explain the terms of behavior from such a perspective. We hope that this will lead to a deeper understanding of both drug dependence and the behavior itself.

See this website: Drug Dependence Concept

Before explaining the terms of behavior, the diagram below shows the basic structure of the establishment of behavior (response). First, living organisms are born with two types of responses. These are spontaneous responses (Emitted Response) that occur even without any specific external stimulus, and induced responses (reflexes) that occur in response to a specific external stimulus. These responses are extremely important responses that living organisms are born with in order to adapt to the environment. In order for the organism to continue to survive in that environment, these alone are not sufficient, and the establishment of learned behavior based on the above responses is necessary. Of the above, learning based on spontaneous responses is called operant responses. On the other hand, learning based on stimulus induced responses is called respondent responses. The majority of living organisms' behaviors are made up of these four types of responses (spontaneous responses -> operant responses, induced responses -> respondent responses), and we believe that understanding responses as having this structure is productive in terms of understanding behavior as a whole. Therefore, the following descriptions of terms related to behavior will be based on this framework.

See other page of this website: Operant Behavior in Neuroscience

3. Behavioral Terminology

Operant Behavior

Drug-seeking behavior in laboratory animals can be formed as lever-pressing operant behavior with an intravenously injected drug as a reinforcing stimulus. On the other hand, there is already a huge amount of studies on the learning behavior of laboratory animals, where they obtain food or juice by lever pressing, etc. (see Operant Behavior in Neuroscience). B. F. Skinner of Harvard University and others have systematized the science and technology of operant behavior, and have shown that not only laboratory animals, but also most of our daily and social behaviors are based on various kinds of operant behaviors.

In general, humans and other animals are premised on the existence of innate spontaneous responses (Emitted Responses) that exist before learning. One of the first behaviors that an infant shows after being born into this world is the intake of milk from its mother. At first, the infant's hesitant contact response to the nipple, etc., gradually becomes an efficient and accurate milk intake response. This becomes one of the first established operant behaviors (lever-pressing responses are not the only operant behavior). Starting from this, the intake of water and food necessary for survival is formed through learning. As is clear from this, the principle of operant behavior is that first, various spontaneously emitted responses occur, and the frequency of occurrence of those that are effective for survival increases. When these occur stably and frequently, learning of various operant behaviors is established. Here, there is a diagram in which various spontaneously emitted responses and environmental stimuli are linked to the specific responses and the specific stimuli in the environment, and if the stimuli are effective for survival, learning is established in this stimulus acquisition response. This is called response contingency, and here is the essence of the establishment of operant behavior. Here, the principle of stimulus reinforcement for a response is important.

In addition to spontaneously emitted responses and the learned behavior of operant behavior, there is another important behavior. It is the elicited response or reflex by the stimulus, and there is a learned behavior of this, called respondent behavior. This stimulus elicited response and respondent behavior are also important aspects of biological behavior. Please refer to the section on "Respondent behavior" below for more information.

In most experimental studies of operant behavior in laboratory animals, food or liquid is used as reinforcer for shaping its behavior. On the other hand, intravenous drug self-administration operant behavior, in which a drug is injected intravenously in response to an animal's lever-pressing response, has become a core research method for studying psychic dependence in drug dependence studies. In other words, when searching for psychis dependence in laboratory animals, drug self-administration method allows observation of drug-seeking behavior, which in turn allows for the search for psychic dependence on drugs (see this website, "Intravenous Drug Self-Administration" in Operant Behavior in Neuroscience and Drug Dependence Concept).

The following is one of the most appropriate textbooks for understanding general concept of operant behavior. This book is structured to be read through by program learning, and each piece of knowledge and concept is firmly learned before moving on to the next step. It is different from normal reading, where you read through the pages whether you understand them or not. As you read this book, you will realize that the reading behavior of this book is supported by the principles of operant behavior, and after reading it, you will be left with a refreshing sense of accomplishment. Although it is a textbook from more than half a century ago, it allows you to properly learn the basics of operant behavior science. In the latest medical/biological fields such as genetic engineering, molecular biology, and immunology, old textbooks may have historical significance, but they may not be sufficient to absorb correct knowledge. In these fields, research subjects are broken down into individual elements and continue to be thoroughly analyzed/clarified, and previous concepts can change significantly. On the other hand, behavioral science research is also advancing day by day, but the idea of ​​behavior, the most advanced and integrated functional framework of a living organism, will not change, as long as it is correct. This is why we believe the following textbook has not faded.

Holland, J.G. and Skinner B.F.: The Analysia of Behavior, A Program for Self-Instruction. McGraw Hill Book Company, Inc. 1961.

The following is also called the bible of operant behavior. It takes some effort to read the whole book. More than 85 years ago, Skinner, a young man, compiled the contents of his dissertation at Harvard University into a book. Of particular note is the section towards the end of the book that describes his joint research with W.T. Heron. In other words, he describes the effect of caffein and bezedrine (amphetamine) in increasing the lever-pressing operant behavior of rats. He writes in particular about the increase in lever-pressing responses to obtain food, despite the fact that amphetamine has an appetite suppressing effect. About 30 years later, mainly in the United States, the academic field of behavioral pharmacology, which encompasses knowledge of drugs and operant behavior, blossomed.

Skinner BF: The Behavior of Organisms: An Experimental Analysis. 1938, Appleton & Century, reprinted by the B. F. Skinner Foundation in 1991 and 1999.

https://psychology.fas.harvard.edu › people › b-f-skinner

The following paper provides a detailed historical overview of the research achievements of the Skinner school and their future, written in Japanese. The PDF can be viewed at the following URL.

https://www.jstage.jst.go.jp/article/sjpr/18/3/18_129/_pdf/-char/en

Masayoshi Sato: Operant Behavior and Experimental Behavioral Analysis - The History and Fate of the Twins - Psychology Review 1975 Vol. 18 No.3, 129-161 (in Japanese).

佐藤方哉：オペラント行動と実験行動分析学 -その双生児の来し方行末 -   心理学評論 1975年 Vol. 18 No.3, 129-161.

See this website Operant Behavior and Neuroscience

Reinforcing Effect

A catheter is placed in the vein of a rhesus monkey or a rat, and a fixed unit dose of drug is injected by the animal's lever-pressing response. If the animal continues to steadily ingest the drug through sustained and frequent lever-pressing responses, it can be said that the drug has a reinforcing effect. In this case, the lever-pressing response is defined as a behavior reinforced by the drug, and learning behavior related to drug seeking and drug intake is established. In this process, there is a branching point where the animal accidentally touches the lever, the drug is injected, and whether or not this reinforces the next lever-pressing behavior. As a result, the frequency of lever-pressing responses increases, and if this stabilizes, drug-seeking behavior is established and the reinforcing effect of the drug is considered to be demonstrated.

In drug self-administration behavior observation, first, a control condition is set in which a drug vehicle such as saline is injected in response to lever pressing. It is confirmed that the number of lever-pressing responses here is at a low level. The presence or absence of a reinforcing effect of the drug is determined by exceeding this control value.

Schedule of Reinforcement

In an intravenous drug self-administration behavior experiment using rhesus monkeys or rats, the first condition is that a fixed dose of drug is injected for each lever-pressing response of the animals. This is the basic continuous reinforcement schedule (Continuous Reinforcement Schedule or Fixed Ratio 1 Schedule).

However, when using drugs with strong reinforcing effects such as cocaine, animals may overdose on the drug due to frequent lever pressing, develop convulsions, and die during the experiment. To avoid this, we usually start with a continuous reinforcement schedule, and after confirming that a certain level of lever pressing has been observed, we then switch to an intermittent schedule of reinforcement to avoid the animal overdosing on the drug and observe the animal's drug self-administration behavior over a long period of time.

Intermittent reinforcement schedules include ratio schedules, which set a fixed ratio for the number of lever presses required for reinforcement. For example, a fixed dose of drug is injected every 10 lever presses is called a fixed ratio 10 schedule. On the other hand, there is also a fixed interval schedule, which sets a fixed time interval. For example, a fixed interval 10-min schedule would inject a drug for the first lever-press response after 10 minutes.

In addition to these, there are also schedule rules that do not set the schedule value as mentioned above, but instead set a variable value for each lever-press response to reinforce it. However, in this case, a certain average value is set, even though the number is variable. For example, there is a variable ratio 10 schedule. With a variable ratio 10 schedule, reinforcement is given every 10 times on average, even though each number is variable. With a variable interval 10-min schedule, reinforcement is given every 10 minutes on average, even though the time is variable.

For explanation of reinforcement schedules in daily life as example, a fixed ratio schedule applies to cases where income is earned according to the amount of production. A variable-interval schedule of reinforcement can be compared to playing a game as hard as you can and randomly winning money or prizes. The probability of a variable-interval can also increase depending on the degree of proficiency. On the other hand, a fixed-interval schedule of reinforcement is like an office worker who works hard every day and receives his monthly salary on a set day of the month, which could be called an office worker schedule. A variable-interval schedule of reinforcement can also be compared to a store where customers come by and buy things one after another. In both cases, money is used as an example of a reinforcing stimulus. Money is one of the most common and powerful reinforcing stimuli in human society, and is called a conditioned generalized reinforcer.

Furthermore, a progressive ratio schedule might start with a Fixed Ratio of 10 for example, and if this produces a stable lever-pressing operant response from the animal, then double that to a Fixed Ratio of 20, and then further to a Fixed Ratio of 40, and so on, gradually increasing the ratio. The ratio at which the animal finally stops pressing the lever is found and used as an index of the strength of the effect of the reinforcing stimulus. 

In behavioral experiments using intravenous drug self-administration with rhesus monkeys, this schedule was used to observe the strength of the reinforcing effects of cocaine, morphine, methamphetamine, nicotine, and other drugs in that order. This corresponds to the strength of the psychic dependence properties of these drugs in humans.

The final ratio of a progressive ratio schedule is defined as the point at which the number of responses at that ratio does not reach the next ratio within a predetermined period of time, and this is called the final ratio (or breaking point). With this progressive ratio schedule, careful consideration is required regarding how the ratio is increased and how the final ratio conditions are determined.

One of the classics on reinforcement schedules is

Ferster, C. B. and Skinner, B. F.: Schedules of reinforcement. Appleton Centre-Crofts, 1957.

Behavioral Extinction

In the above experiment on self-administration of drugs, the subject is first given a vehicle such as saline in response to the lever-pressing response. Here, it is first confirmed that there is no clear number of times of ingestion. This level is called the operant level. Then, the subject is allowed to take a drug such as cocaine. After a certain number of times of ingestion is observed and the reinforcing effect of the drug is confirmed, the subject is returned to the condition of giving the vehicle such as saline in response to the lever-pressing response. At this stage, when the drug is switched to the vehicle, a high frequency of lever-pressing responses, called a burst, is observed at first. However, the lever-pressing response eventually reaches the same operant level as the initial vehicle condition. In this way, the decrease in the response when taking the vehicle after searching for the reinforcing effect of the drug is said to be an extinction of the lever-pressing behavior.

When extinction of the behavior is not easily observed, it is said that there is strong extinction resistance. In the progressive ratio reinforcement schedule for self-administration of drugs mentioned above, the strength of extinction resistance of each drug is measured. In other words, the more difficult it is to extinguish a drug and the stronger the extinction resistance, the stronger the degree of exploratory behavior for that drug is considered to be. Furthermore, the strength of this extinction resistance can be considered as a behavioral expression of compulsive craving for drugs in humans. Therefore, by using such a progressive ratio reinforcement schedule, it is possible to observe intense drug-seeking behavior equivalent to compulsive craving for drugs in animal experiments, and therefore it is believed that psychic dependence on the drug can be detected with high validity. The problem with drug dependence is that the drug-seeking behavior is in a state of strong extinction resistance.

While the strength and degree of the reinforcing effect of the drug is examined under progressive ratio reinforcement schedule conditions, it can be said that drug self-administration behavior under a simple continuous reinforcement schedule is merely examining the presence or absence of the reinforcing effect of the drug. The existence of a reinforcing effect of a drug is one prerequisite for the existence of psychic dependence on the drug. However, strictly speaking, this alone does not clarify the presence or absence of psychic dependence on the drug. Therefore, in order to investigate psychic dependence on drugs more deeply, it would be theoretically important to observe compulsive drug-seeking behavior under a progressive ratio schedule, not just continuous reinforcement schedules and intermittent reinforcement schedules. In other words, it is necessary to measure the strength of extinction resistance of the reinforcing effect of the drug.

Stimulus Discrimination

When the drug self-administration behavior under the above-mentioned reinforcement schedule is stably maintained, a condition is set in which, for example, when the lamp is on, the response is reinforced, and when the lamp is off, the response is not reinforced. Through repeated training, the animal will show drug self-administration behavior when the lamp is on, and will not show this behavior when the lamp is off. In other words, the animal will show an appropriate response corresponding to the presence or absence of the external stimulus of the lamp being on. This is called the animal's stimulus discrimination behavior. The presence or absence of the lamp plays the role of a discriminative stimulus. If the appropriate experimental conditions are set, the animal will be able to discriminate between the presence or absence of these stimuli, not only visual stimuli such as lamps, but also auditory and olfactory stimuli. As a result, there are two aspects: the aspect that discrimination was observed in the animal's behavior, and the aspect that the animal discriminated (perceived) the stimulus. Even in animals that cannot use verbal instructions like humans, the perception of sensory stimuli and the threshold of sensory stimuli can be measured in detail through such discrimination behavior.

As another example of stimulus discrimination, a drug discrimination experiment will be described. Here, animals are asked to discriminate the internal sensory stimuli when taking an administered drug, without using external sensory stimuli such as the presence or absence of a light as a clue. This applies to both rats and monkeys. In an experimental setting with two levers, for example, after subcutaneous administration of methamphetamine, food is reinforced for pressing the left lever, and on another day, after subcutaneous administration of saline, food is reinforced for pressing only the right lever. By repeating this training for several weeks, the animals will be able to discriminate the difference in internal sensations after administration of methamphetamine and saline by the difference in the left and right lever pressing responses. This method is a useful animal experimental method for investigating the subjective effects after drug administration in humans. The subjective effects of drugs are deeply related to the qualitative aspects of the formation of psychic dependence on drugs. Therefore, it can be said that drug discrimination experiments are closely related in content and methodology to the drug self-administration experiment.

Stimulus Generalization

The discrimination behavior described above is a behavioral characteristic in which an organism discriminates sensory stimuli in great detail. On the other hand, however, there is flexibility in the discrimination. This behavioral characteristic is called stimulus generalization. For example, when human speech is recorded with a microphone and analyzed acoustically, it is perceived as one word, even though there are various physical differences between people. Even if the characteristics of the physical air vibrations are different, similar air vibrations within a certain range will be the same word. In another example, the meanings of red, yellow, and green on traffic lights are probably the same in every country. However, the physical wavelength characteristics of each color differ slightly from country to country. Even if the physical characteristics are slightly different, once they are recognized by humans as color vision, they play the role of stimuli that control the same behavior. Thanks to the mechanisms of stimulus generalization and stimulus discrimination in the brain, living organisms can adapt to their environment accurately and flexibly.

Let us discuss the relationship between stimulus generalization and the previous example of drug discrimination. When an animal that discriminated methamphetamine was administered the generalization test drug cocaine, it chose the lever on the methamphetamine side. It can be said that the internal stimulant effect of methamphetamine generalizes to the effect of cocaine, which is also a central nervous system stimulant. Even when testing with central nervous system depressants such as chlorpromazine, the lever for methamphetamine is not selected but the lever for saline. This means that the central nervous system stimulants and depressants are discriminated from each other, and generalization is not observed between the two.

Conditioning Process

Most of the behaviors described so far can be said to be learned by living organisms through interaction with environmental stimuli. In other words, conditioning is the process in which spontaneous responses and induced responses are learned and each is established as an operant response and a respondent response, respectively. There is no doubt that neural networks corresponding to each are formed in the brain. There will be a future in which the full content of this will be revealed by neuroscience. In addition, the learning process is being imitated or elucidated by computers using deep learning, and we hope that the content of the conditioning process will also be revealed in this area.

Conditioned Stimulus

In the above conditioning process, the part of the environmental stimulus that has no inherent relationship to a specific response but plays a role that is uniquely and deeply related to a certain response is the conditioned stimulus, and the response is controlled by this conditioned stimulus. However, the conditioned stimulus is originally neutral and unrelated to the occurrence of the response, and through conditioning, a process is established in which the response has a unique relationship with the conditioned stimulus. At this stage, the conditioned stimulus exhibits both the phenomena of stimulus discrimination and stimulus generalization. However, even when the conditioned stimulus is presented, if the response is not continuously reinforced, extinction occurs and the conditioned stimulus no longer controls the response.

Positive Reinforcement / Negative Reinforcement / Punishment

As a behavioral science classification of learning or conditioning, first of all, there are behaviors based on positive reinforcement and negative reinforcement. So far, we have mainly discussed positive reinforcement using examples such as food and juice intake behavior of laboratory animals and drug self-administration behavior. This refers to a situation in which operant responses such as food, juice, or intravenously injected drugs are maintained frequently and continuously. On the other hand, responses due to negative reinforcement refer to a situation in which the animal escapes or avoids stimuli such as electric shocks. An escape response is a response in which the animal escapes from the shock by pressing a lever while receiving the shock. On the other hand, in an avoidance response, a conditioned stimulus such as a buzzer is presented as a warning sound before the shock is received. Through repeated training, the animal will show an avoidance response without receiving the shock when the warning sound is presented.

In the above cases, the animal can escape or avoid the electric shock through learning, but punishment refers to a situation in which neither escape nor escape is possible. An example of a situation in which an electric shock is presented in operant behavior in which an animal obtains food by pressing a lever under positive reinforcement conditions is a situation in which an electric shock is presented in synchronization with or asynchronously with the lever press. During this period, a conditioned stimulus such as a lamp or buzzer is usually presented. Under such circumstances, the animal's response for positive reinforcement is suppressed during the presentation of the conditioned stimulus. However, if the animals are given an anti-anxiety drug such as diazepam, a benzodiazepine derivative, the response inhibition caused by the electric shock is removed, and the animals continue to respond despite receiving the shock, demonstrating the disinhibition effect of the drug.

Repeated intake of some of the many types of drugs that act on the central nervous system can lead to the formation of a state of drug dependence in the body (see the diagram above). Drug dependence can be classified into psychic dependence and physical dependence. Psychic dependence plays a major role in the formation and maintenance of drug dependence. This is characterized by a compulsive craving for the drug, and it is believed that such a structure is formed and established in the neural network in the brain. In animal experiments, drug-seeking behavior can be observed through intravenous/intragastric drug self-administration, which is similar to the state of psychic dependence in humans. On the other hand, a separate state of physical dependence can also be formed in the body as a result of repeated intake based on psychic dependence. Physical dependence, also known as neuroadaptation, is one of the physiological adaptive phenomena of the body that appears as a result of repeated drug intake. The formation of physical dependence can be detected by the presence or absence of withdrawal syndrome when repeated drug intake is discontinued. Depending on the type of drug, these can be severe syndrome such as vomiting, convulsions, and sweating. Note that in this diagram, physical dependence is described as a subset of psychic dependence. This is because not all dependence producing drugs lead to physical dependence. Furthermore, through repeated drug intake as described above, changes in sensitivity to drugs may occur in the body although it is not the core of dependence. This is a tolerance or sensitization effect to the drug, and has a significant impact on drug dependence.

On the other hand, repeated intake of non-frprndence producing drugs may also cause the body to adapt to the phenomenon equivalent to physical dependence. This is manifested as the appearance of withdrawal syndrome for the non-dependence producing drug. For example, if a patient is taking steroids (corticosteroids) used for anti-inflammation and immunosuppression for treatment and suddenly stops taking them, withdrawal syndrome such as worsening inflammation and primary adrenal insufficiency occur. In such cases, the main effect of the drug is not on the central nervous system, and psychic dependence is not the main issue, so they are excluded from the framework of drug dependence in the diagram.

See this website: Drug Dependence Concept & Operant Behavior in Neuroscience