| Date | Weight (g) |
| 12-Sep | 235 |
| 13-Sep | 159 |
| 14-Sep | 220 |
| 15-Sep | 210 |
| 16-Sep | 215 |
| 17-Sep | 208 |
| 18-Sep | 205 |
| 19-Sep | 205 |
| 20-Sep | 210 |
| 21-Sep | 211 |
| 22-Sep | 212 |
| 23-Sep | 208 |
| 24-Sep | 208 |
| 25-Sep | 208 |
| 26-Sep | 208 |
| 27-Sep | 205 |
| 28-Sep | 210 |
| 29-Sep | 204 |
| 30-Sep | 203 |
| 1-Oct | 295 |
| 2-Oct | 206 |
| 3-Oct | 207 |
| 4-Oct | 205 |
| 5-Oct | 208 |
| 6-Oct | 202 |
Wednesday, October 10, 2012
Weight Chart
Target Weight: 204 grams
Conclusion
My favorite part about this experience was definitely working with my rat, Li'l Swanson. I was surprised by how intelligent she is and how quickly she learned to press the bar--if you had asked me a year or two ago if I thought it would be possible for me to train a rat to press a bar, I would have said no. Now, however, I realize how relatively easy it is if you just use shaping. I am now much more confident in my training abilities, and I'm really excited to get to train my friend Kelsie's dog, Kya, for my next project! I do not have any suggestions for improvement, except perhaps to give more explicit instructions for how to print/post the cumulative records off of the lab computers--it took me a while to figure that out. I don't think I began this experience with any misconceptions, other than the misconception that training a rat to press a bar would be really difficult, which I quickly learned was not the case. This project has made me realize how applicable (and fun!) the principles of learning can actually be.
Problems in Paradise
Li'l Swanson was an excellent rat. During magazine training she associated the sound with food arriving in the magazine pretty quickly, during shaping she exhibited a wide enough variety of behaviors that I was able to make progress with her during each training session, and during the various schedules of reinforcement, she made it all the way to the FR10 schedule. I really had almost no problems with her except for one, fairly insignificant behavior that I was never able to stop: Li'l Swanson constantly sniffed one particular corner (the front left corner) of the operant box, possibly because she could smell other rats that had been in the box earlier. This would not have been much of a problem (it was just sniffing, after all) except that she would sniff for minutes at a time, which would keep her from pressing the bar. I was very careful to never reward this behavior, and I was able to decrease it by cleaning out the operant box before each time I put Swanson in it. She still fixated on that spot even after I wiped down everything in the box with soap and water, but she did stay in that one corner for a shorter amount of time during each session. I do not think there is anything I could have done differently to solve the problem except maybe clean even more than I did.
Looking back, the only way I can see myself improving from this experience is in shaping. Now that I have some experience and understand what does and does not work, I think I could now shape a rat to press a bar in a shorter amount of time than I did for this project (it took me 4 days, or about 2 hours total). I think I could be more efficient and more particular about which successive approximations of the desired behavior I would reinforce. Other than the shaping, though, I don't think there is much else I could have improved on, thanks to your instructions and having such a great rat :)
Sniffy
*Dr. Trench: This post includes both the "Sniffy" post and the "Real Rat vs. Virtual Rat" post
The small golden-brown nugget-like object that Sniffy is sniffing is the reward (food) and it is sitting in the box's magazine. Directly above the magazine is the lever that I shaped Sniffy to press. Just like Li'l Swanson, I had to first magazine train Sniffy (train him to associate a particular sound with food arriving in the magazine), then shape Sniffy by using that learned association to reward him for performing successive approximations of pressing the lever. Magazine training Sniffy took me about 25 minutes, and shaping him took me about 45 minutes. Magazine training Sniffy and Swanson were very similar, and both fairly easy. They both took around the same amount of time (20-25 minutes); the biggest difference was the number of times I had to activate the magazine--I had to feed Sniffy many more pellets before he associated the noise and food than I had to feed Swanson. Because I had never trained a rat before, I found shaping Sniffy to be more difficult than shaping Li'l Swanson even though it took me much less time to shape Sniffy (45 minutes vs. 2-ish hours). Shaping Li'l Swanson was probably easier because I somewhat knew what to expect because of my practice with Sniffy. It was also more difficult to shape Sniffy because it was hard to tell sometimes when Sniffy was performing one of the successive approximations of the target behavior--for example, I began by reinforcing Sniffy whenever he faced the bar, but sometimes the computer would lag for a second and I wouldn't get the timing of the reinforcing perfectly right, or it would be difficult to tell if Sniffy was actually facing the bar (his body would be pointed towards it but his face would be at a slight angle away from the bar), and I had a hard time determining if the computer recognized that I was rewarding the desired behavior.
After Sniffy was completely shaped (was pressing the bar voluntarily and often in order to receive food), I put him on a VI5 schedule of reinforcement. A VI5 schedule, or variable interval 5 schedule, reinforces the rat when it presses the bar for the first time after an interval of about 5 seconds has passed (the interval could actually be anywhere from about 3-7 seconds, but the average amount of time that would have to pass before Sniffy could get rewarded for pressing the bar was 5 seconds). This is Sniffy's cumulative record for the VI5 schedule:
And this is Swanson's cumulative record for FR5, to compare:
Notice how, at the beginning, Sniffy pressed the bar over and over, expecting a reward. However, after trial and error--pressing the bar and not getting rewarded unless about 5 seconds had passed since the last reward, Sniffy eventually started pressing the bar about every 5 seconds, causing the cumulative record increase at a constant rate (see 2nd image above). Li'l Swanson's bar pressing is more irregular than Sniffy's bar pressing (after he had gotten used to the schedule, that is). This difference could be simply because of the variations in Swanson's behavior because Swanson is a real rat, or because of the differences between fixed ratio and variable interval schedules.
After Sniffy had mastered the VI5 schedule of reinforcement, I bumped him up to a VI10 schedule of reinforcement, where he would get reinforced the first time he pressed the bar after approximately 10 seconds had passed since the last reinforcement. This is the VI10 cumulative record:
Notice how, again, as time goes on, Sniffy's responses (bar presses) become more regular as he learns the schedule of reinforcement.
There are both strengths and weaknesses of using live and virtual rats. The virtual rats are much easier to deal with--you don't have to feed them, or weigh them, or change their bedding. However, the computer program does not factor in the differing personalities you get in different rats, or the variations in the rats' behavior. Training Sniffy was fairly straightforward, and training Swanson was a little more complicated--I had to make sure that I didn't train Swanson to fixate too much on the light in the box, which wasn't something that Sniffy would have done. I think Sniffy was a perfect training tool to practice on before we trained our real rats, and I would recommend continuing to use both--I think that the experience you get training Sniffy before your real rat is invaluable.
Extinction
The last two days of training Swanson consisted of extinction--ceasing the rewards in order to stop the behavior. I put Li'l Swanson in the operant box, and for the full 30 minutes she was not rewarded one single time that she pressed the bar. The chart below shows how many times she pressed the bar in each 5-minute interval:
As you can see, as time went on, she pressed the bar fewer and fewer times. She often exhibited odd behaviors during extinction--hopping around the cage, chewing on the bar, sniffing the bar then the magazine then the bar again, etc., as if she was frustrated that there were no rewards appearing in the magazine. During day 2 of extinction, at the 20 minute mark, her number of bar presses doubled (from 11 to 23) after almost constantly decreasing in each 5-minute interval. This could be considered an example of extinction burst, albeit a weak example. I did not see any examples of spontaneous recovery.
In the video below, notice how Li'l Swanson searches inside the (empty) magazine for food, then presses the bar repeatedly, then chews on the bar (probably in frustration), then continues pressing the bar.
| Time (min) | Bar presses | |
| Extinction Day 1 | 5 | 79 |
| 10 | 57 | |
| 15 | 31 | |
| 20 | 13 | |
| 25 | 5 | |
| 30 | 13 | |
| Extinction Day 2 | 5 | 30 |
| 10 | 25 | |
| 15 | 11 | |
| 20 | 23 | |
| 25 | 18 | |
| 30 | 3 |
As you can see, as time went on, she pressed the bar fewer and fewer times. She often exhibited odd behaviors during extinction--hopping around the cage, chewing on the bar, sniffing the bar then the magazine then the bar again, etc., as if she was frustrated that there were no rewards appearing in the magazine. During day 2 of extinction, at the 20 minute mark, her number of bar presses doubled (from 11 to 23) after almost constantly decreasing in each 5-minute interval. This could be considered an example of extinction burst, albeit a weak example. I did not see any examples of spontaneous recovery.
In the video below, notice how Li'l Swanson searches inside the (empty) magazine for food, then presses the bar repeatedly, then chews on the bar (probably in frustration), then continues pressing the bar.
FR Schedule of Reinforcement and Chart of Responses per 30 Minutes
After Li'l Swanson had been successfully shaped, I began putting her on different schedules of reinforcement. Because rewarding an organism every single time for performing a desired action is not particularly practical, you can reward the organism in different patterns, which elicit different amounts of behavior in the organism. I began Li'l Swanson on the FR2 (fixed ratio 2) schedule of reinforcement, which means that I reinforced her for every two times she pressed the bar. At the beginning of this training session, she would look for a pellet in the magazine after every single time she pressed the bar, even though there would only be a pellet there every other time she pressed the bar.
However, as time passed, Li'l Swanson eventually stopped checking the magazine every single time (although occasionally she returned to that behavior) and would only check it after she had pressed the bar twice. At the end of this session, Li'l Swanson had pressed the bar about 86 times.
The next day, I put Li'l Swanson on an FR3 schedule of reinforcement. This schedule works the same way the FR2 schedule does--reinforcing the behavior after a certain number of times the behavior is performed--except Swanson was reinforced for every three times she pressed the bar. Notice how the cumulative record below differs from the cumulative record above:
Reinforcing Swanson less often led to her pressing the bar many more times. On the FR3 schedule of reinforcement, Swanson pressed the bar a total of 184 times, more than double the number of times she pressed the bar on the FR2 schedule.
The next training day I moved Swanson up to a FR5 schedule, where she was reinforced for every 5 times she pressed the bar, and she pressed the bar 272 times. The next day I put her on the FR7 schedule, where she was reinforced every 7 times she pressed the bar, and she pressed it 276 times. The day after that (my 10th day of training, including the magazine training and shaping), I moved Swanson up to the FR10 schedule of reinforcement, where she was reinforced for every 10 times she pressed the bar. If she had continued in her pattern of pressing the bar more when she was on a higher schedule of reinforcement, I would have expected her to press the bar more than 270 times. However, this was not the case. Li'l Swanson only pressed the bar 94 times when I put her on the FR10 schedule (see graph below)
As you can see, Swanson pressed the bar fairly consistently (albeit not too often) during the first 15 minutes or so, but after that she only pressed the bar a few more times. I ended the session early because she seemed to have "given up." Because of her low performance, the next training day I put her back on the FR10 schedule of reinforcement, and she did much better, with a total of 252 bar presses:
Because Li'l Swanson had not increased her number of bar presses much from FR5 to FR7 to FR10, I decided to put her on a different schedule for my last two days of reinforcement--a variable reinforcement schedule. In this schedule, the organism gets reinforced every time they perform an action for a varying number of times. For example, in a VR3 schedule of reinforcement, Swanson would get reinforced about every 2, 3, or 4 times she pressed the bar, but the average number of times she would have to press the bar for a reward would be 3 times. I used the VR3 schedule next, and she did well--232 bar presses. The next day, however, was her personal best: on the VR 5 schedule, she pressed the bar 407 times:
However, as time passed, Li'l Swanson eventually stopped checking the magazine every single time (although occasionally she returned to that behavior) and would only check it after she had pressed the bar twice. At the end of this session, Li'l Swanson had pressed the bar about 86 times.
The next day, I put Li'l Swanson on an FR3 schedule of reinforcement. This schedule works the same way the FR2 schedule does--reinforcing the behavior after a certain number of times the behavior is performed--except Swanson was reinforced for every three times she pressed the bar. Notice how the cumulative record below differs from the cumulative record above:
The next training day I moved Swanson up to a FR5 schedule, where she was reinforced for every 5 times she pressed the bar, and she pressed the bar 272 times. The next day I put her on the FR7 schedule, where she was reinforced every 7 times she pressed the bar, and she pressed it 276 times. The day after that (my 10th day of training, including the magazine training and shaping), I moved Swanson up to the FR10 schedule of reinforcement, where she was reinforced for every 10 times she pressed the bar. If she had continued in her pattern of pressing the bar more when she was on a higher schedule of reinforcement, I would have expected her to press the bar more than 270 times. However, this was not the case. Li'l Swanson only pressed the bar 94 times when I put her on the FR10 schedule (see graph below)
As you can see, Swanson pressed the bar fairly consistently (albeit not too often) during the first 15 minutes or so, but after that she only pressed the bar a few more times. I ended the session early because she seemed to have "given up." Because of her low performance, the next training day I put her back on the FR10 schedule of reinforcement, and she did much better, with a total of 252 bar presses:
Because Li'l Swanson had not increased her number of bar presses much from FR5 to FR7 to FR10, I decided to put her on a different schedule for my last two days of reinforcement--a variable reinforcement schedule. In this schedule, the organism gets reinforced every time they perform an action for a varying number of times. For example, in a VR3 schedule of reinforcement, Swanson would get reinforced about every 2, 3, or 4 times she pressed the bar, but the average number of times she would have to press the bar for a reward would be 3 times. I used the VR3 schedule next, and she did well--232 bar presses. The next day, however, was her personal best: on the VR 5 schedule, she pressed the bar 407 times:
So looking back at the number of bar presses Li'l Swanson did on each schedule, the VR5 schedule elicited the most.
The following is a chart of the number of bar presses per 30 minutes for each of the different schedules (and each day of shaping) that I used with Li'l Swanson:
| Schedule | Number of Responses per 30 minutes |
| Shaping 1 | 7.2 |
| Shaping 2 | 2.6 |
| Shaping 3 | 7 |
| Shaping 4 | 67 |
| FR2 | 107.5 |
| FR3 | 204.4 |
| FR5 | 272 |
| FR7 | 276 |
| FR10 | 141 |
| FR10 pt. 2 | 252 |
| VR3 | 257.8 |
| VR5 | 407 |
Thursday, September 27, 2012
Shaping: Day 4
As soon as I put Li'l Swanson in the operant box, she went straight to the bar and placed her paws on it, then looked for a reward. I rewarded her the first few times that she placed her paws on the bar, but after the first few minutes I switched to only rewarding her every 3 or 4 times she placed her paws on the bar in the hopes that she would get frustrated and press harder on the bar, my ultimate goal. She started pressing on the bar after she'd been in the box for about 8 minutes, and by the end of the 30 minutes session she was pressing the bar consistently--she had been shaped! Overall, the shaping process went as I expected, based on our class readings. By using "successive approximations" of the desired behavior, I trained Li'l Swanson to press the bar--trying to get her to press the bar from the get-go would have been nearly impossible, but by rewarding behavior in small steps, I was able to accomplish my task.
In the graph above, the x-axis represents time passed, and the y-axis represents lever presses. As you can see, at the beginning of this session with Li'l Swanson, she did not press the bar very often. However, as time went on, her bar presses increased greatly--by the end of the shaping session, she pressed the bar about 67 times.
In the graph above, the x-axis represents time passed, and the y-axis represents lever presses. As you can see, at the beginning of this session with Li'l Swanson, she did not press the bar very often. However, as time went on, her bar presses increased greatly--by the end of the shaping session, she pressed the bar about 67 times.
Subscribe to:
Posts (Atom)