;=== Evoloution of Game-Playing Agents === ;=== by Jim Lyons === December 1, 2004 === globals [num-games player-colors] breeds [players games things] games-own [heap-patches heaps player-patches player1 player2] players-own [free game book record num-won num-lost] ;=== OBSERVER === to startup clear-all set player-colors [3 14 24 33 103 63 113 133] set-default-shape things "box" set-default-shape players "person" ; make a pretty background ask patches [ set pcolor brown + 1 + random-float .6 ] diffuse pcolor .3 ; make players first so "who" will be 0-7 create-custom-players 8 [ set color item who player-colors setxy who * 5 - 17 -10 set size 5 set free true set book [ [] [] ] set record [] set-label ] ; make the games ask patch -21 4 [ sprout 1 [set breed games] ] ask patch -7 4 [ sprout 1 [set breed games] ] ask patch 7 4 [ sprout 1 [set breed games] ] ask patch 21 4 [ sprout 1 [set breed games] ] ask games [ make-game ] setup-plots end to play-games ask games [ new-game ] ask games [ play-game ] set num-games num-games + 1 do-plots end to setup-plots set-current-plot "Book Size" foreach n-values 8 [?] ; player who numbers [ set-current-plot-pen ("" + ?) set-plot-pen-color item ? player-colors plot 0 ] end to do-plots set-current-plot "Book Size" foreach n-values 8 [?] ; player who numbers [ set-current-plot-pen ("" + ?) plot length item 0 book-of turtle ? ] end ;=== PATCHES === to grow-heap [n] ; called by make-game to grow heap and make things set pcolor pcolor + .3 ; creates hint of heap location and size sprout 1 ; make thing here and hide it (things are unhidden when needed) [ set breed things set color orange + 1 set heading 0 set size 1.25 set hidden? true ] set n n - 1 ; continue growing to top of heap if n > 0 [ ask patch-at 0 1 [ grow-heap n ] ] end ;=== GAMES === to make-game set hidden? true ask patches with [ abs (pxcor - xcor-of myself) <= 5 and abs (pycor - ycor-of myself) <= 5 ] [ set pcolor brown + 3 ] ; make game area if num-heaps = 3 [ set heap-patches (list patch-at -3 -4 patch-at 0 -4 patch-at 3 -4) ] if num-heaps = 4 [ set heap-patches (list patch-at -3 -4 patch-at -1 -4 patch-at 1 -4 patch-at 3 -4) ] if num-heaps = 5 [ set heap-patches (list patch-at -4 -4 patch-at -2 -4 patch-at 0 -4 patch-at 2 -4 patch-at 4 -4) ] set player-patches list patch-at -3 -8 patch-at 2 -8 foreach heap-patches [ ask ? [ grow-heap max-things ] ] end to new-game locals [n] set heaps [] ; build new counts list foreach heap-patches [ without-interruption ; because of local n [ set n 1 + random max-things if animate? [ ask things-on ? [set-things n] ] set heaps lput n heaps ] ] ; get two players without-interruption [ ask random-one-of players with [free] [ set game myself set player1-of myself self set free false ] ask random-one-of players with [free] [ set game myself set player2-of myself self set free false ] ] if animate? ; move players here [ ask player1 [ move-to item 0 player-patches-of myself ] ask player2 [ move-to item 1 player-patches-of myself ] ] end to play-game loop [ make-move value-from player1 [your-move] if sum heaps = 0 [ ask player1 [ you-win ] ask player2 [ you-lose ] stop ] make-move value-from player2 [your-move] if sum heaps = 0 [ ask player2 [ you-win ] ask player1 [ you-lose ] stop ] ] end to make-move [move] ; move is a two-digit string... locals [new-count from-heap index] set from-heap read-from-string item 1 move ; second digit is which heap, its count set new-count from-heap - read-from-string item 0 move ; first is how many to take set index position from-heap heaps if is-number? index [ set heaps replace-item index heaps new-count if animate? [ ask things-on (item index heap-patches) [ set-things new-count ] wait .1 ] ] end to-report random-move locals [which taken] set which random num-heaps while [item which heaps = 0] [ set which ((which + 1) mod num-heaps) ] set taken 1 + random (item which heaps) report "" + taken + item which heaps end ;=== THINGS === to set-things [n] ; show things up to n, hide the rest set n n - 1 set hidden? (n < 0) ask things-at 0 1 [ set-things n ] end ;=== PLAYERS === to-report your-move locals [pos move] set pos heaps-code heaps-of game set move read-book pos if move = "" [ set move value-from game [random-move] ] set record lput list pos move record report move end to you-win if animate? [ set ycor -9 ] ; enter moves from this game in the book foreach record [ enter-move item 0 ? item 1 ? ] set record [] set num-won num-won + 1 set-label set free true end to you-lose if animate? [ set ycor -10 ] ; delete moves from book foreach record [ remove-entry item 0 ? item 1 ? ] set record [] set num-lost num-lost + 1 set-label set free true end to move-to [some-agent] if some-agent = self [ stop ] set heading towards-nowrap some-agent while [distance some-agent > 1] [ fd 1 wait .03 ] setxy pxcor-of some-agent pycor-of some-agent end to set-label if labels = "None" [ set label no-label ] if labels = "Number" [ set label who ] if labels = "Book" [ set label length item 0 book ] if labels = "Won/Lost" [ set label "" + int num-won + "/" + int num-lost ] end to-report read-book [ code ] locals [ index ] set index position code (item 0 book) ; look up key if is-number? index [ report item index (item 1 book) ] report "" ; not in the book end to enter-move [ code move ] locals [ index ] set index position code (item 0 book) ; look up key ifelse is-number? index [ set book list (item 0 book) replace-item index (item 1 book) move ] [ set book list lput code (item 0 book) lput move (item 1 book) ] end to remove-entry [ code move ] locals [ index ] set index position code item 0 book if is-number? index [ if item index item 1 book = move [ set book list remove-item index (item 0 book) remove-item index (item 1 book) ] ] end to-report heaps-code [ heap-list ] ; report normalized code for given heaps locals [code] set heap-list sort heap-list ; normalize set code "" foreach heap-list [ set code code + ? ] report code end @#$#@#$#@ GRAPHICS-WINDOW 116 10 639 248 28 11 9.0 1 10 1 1 1 CC-WINDOW 5 479 648 574 Command Center CHOICE 11 10 105 55 num-heaps num-heaps 3 4 5 0 CHOICE 11 55 105 100 max-things max-things 5 6 7 8 9 1 BUTTON 11 100 105 133 New Run... if user-yes-or-no? "This will clear all players' books. Proceed?"\n[startup] NIL 1 T OBSERVER T SWITCH 12 137 106 170 animate? animate? 1 1 -1000 BUTTON 12 215 106 248 Play Games play-games T 1 T OBSERVER T CHOICE 12 170 106 215 labels labels "None" "Number" "Book" "Won/Lost" 2 MONITOR 12 253 106 302 NIL num-games 0 1 PLOT 346 252 639 465 Book Size Number of Games Number of Entries 0.0 250.0 0.0 50.0 true true PENS "0" 1.0 0 -16776961 true "1" 1.0 0 -16711681 true "2" 1.0 0 -16745473 true "3" 1.0 0 -16776961 true "4" 1.0 0 -8716033 true "5" 1.0 0 -65281 true "6" 1.0 0 -65413 true "7" 1.0 0 -16777216 true TEXTBOX 133 261 312 314 WARNING: Only turn animate on or off when play is stopped or the run may have to be aborted. @#$#@#$#@ NIM Players This model demonstrates how agents can learn to play a game by trial and error. @#$#@#$#@ default true 0 Polygon -7566196 true true 150 5 40 250 150 205 260 250 ant true 0 Polygon -7566196 true true 136 61 129 46 144 30 119 45 124 60 114 82 97 37 132 10 93 36 111 84 127 105 172 105 189 84 208 35 171 11 202 35 204 37 186 82 177 60 180 44 159 32 170 44 165 60 Polygon -7566196 true true 150 95 135 103 139 117 125 149 137 180 135 196 150 204 166 195 161 180 174 150 158 116 164 102 Polygon -7566196 true true 149 186 128 197 114 232 134 270 149 282 166 270 185 232 171 195 149 186 Polygon -7566196 true true 225 66 230 107 159 122 161 127 234 111 236 106 Polygon -7566196 true true 78 58 99 116 139 123 137 128 95 119 Polygon -7566196 true true 48 103 90 147 129 147 130 151 86 151 Polygon -7566196 true true 65 224 92 171 134 160 135 164 95 175 Polygon -7566196 true true 235 222 210 170 163 162 161 166 208 174 Polygon -7566196 true true 249 107 211 147 168 147 168 150 213 150 arrow true 0 Polygon -7566196 true true 150 0 0 150 105 150 105 293 195 293 195 150 300 150 bee true 0 Polygon -256 true false 152 149 77 163 67 195 67 211 74 234 85 252 100 264 116 276 134 286 151 300 167 285 182 278 206 260 220 242 226 218 226 195 222 166 Polygon -16777216 true false 150 149 128 151 114 151 98 145 80 122 80 103 81 83 95 67 117 58 141 54 151 53 177 55 195 66 207 82 211 94 211 116 204 139 189 149 171 152 Polygon -7566196 true true 151 54 119 59 96 60 81 50 78 39 87 25 103 18 115 23 121 13 150 1 180 14 189 23 197 17 210 19 222 30 222 44 212 57 192 58 Polygon -16777216 true false 70 185 74 171 223 172 224 186 Polygon -16777216 true false 67 211 71 226 224 226 225 211 67 211 Polygon -16777216 true false 91 257 106 269 195 269 211 255 Line -1 false 144 100 70 87 Line -1 false 70 87 45 87 Line -1 false 45 86 26 97 Line -1 false 26 96 22 115 Line -1 false 22 115 25 130 Line -1 false 26 131 37 141 Line -1 false 37 141 55 144 Line -1 false 55 143 143 101 Line -1 false 141 100 227 138 Line -1 false 227 138 241 137 Line -1 false 241 137 249 129 Line -1 false 249 129 254 110 Line -1 false 253 108 248 97 Line -1 false 249 95 235 82 Line -1 false 235 82 144 100 bird1 false 0 Polygon -7566196 true true 2 6 2 39 270 298 297 298 299 271 187 160 279 75 276 22 100 67 31 0 bird2 false 0 Polygon -7566196 true true 2 4 33 4 298 270 298 298 272 298 155 184 117 289 61 295 61 105 0 43 boat1 false 0 Polygon -1 true false 63 162 90 207 223 207 290 162 Rectangle -6524078 true false 150 32 157 162 Polygon -16776961 true false 150 34 131 49 145 47 147 48 149 49 Polygon -7566196 true true 158 33 230 157 182 150 169 151 157 156 Polygon -7566196 true true 149 55 88 143 103 139 111 136 117 139 126 145 130 147 139 147 146 146 149 55 boat2 false 0 Polygon -1 true false 63 162 90 207 223 207 290 162 Rectangle -6524078 true false 150 32 157 162 Polygon -16776961 true false 150 34 131 49 145 47 147 48 149 49 Polygon -7566196 true true 157 54 175 79 174 96 185 102 178 112 194 124 196 131 190 139 192 146 211 151 216 154 157 154 Polygon -7566196 true true 150 74 146 91 139 99 143 114 141 123 137 126 131 129 132 139 142 136 126 142 119 147 148 147 boat3 false 0 Polygon -1 true false 63 162 90 207 223 207 290 162 Rectangle -6524078 true false 150 32 157 162 Polygon -16776961 true false 150 34 131 49 145 47 147 48 149 49 Polygon -7566196 true true 158 37 172 45 188 59 202 79 217 109 220 130 218 147 204 156 158 156 161 142 170 123 170 102 169 88 165 62 Polygon -7566196 true true 149 66 142 78 139 96 141 111 146 139 148 147 110 147 113 131 118 106 126 71 box true 0 Polygon -7566196 true true 45 255 255 255 255 45 45 45 butterfly1 true 0 Polygon -16777216 true false 151 76 138 91 138 284 150 296 162 286 162 91 Polygon -7566196 true true 164 106 184 79 205 61 236 48 259 53 279 86 287 119 289 158 278 177 256 182 164 181 Polygon -7566196 true true 136 110 119 82 110 71 85 61 59 48 36 56 17 88 6 115 2 147 15 178 134 178 Polygon -7566196 true true 46 181 28 227 50 255 77 273 112 283 135 274 135 180 Polygon -7566196 true true 165 185 254 184 272 224 255 251 236 267 191 283 164 276 Line -7566196 true 167 47 159 82 Line -7566196 true 136 47 145 81 Circle -7566196 true true 165 45 8 Circle -7566196 true true 134 45 6 Circle -7566196 true true 133 44 7 Circle -7566196 true true 133 43 8 circle false 0 Circle -7566196 true true 35 35 230 person false 0 Circle -7566196 true true 155 20 63 Rectangle -7566196 true true 158 79 217 164 Polygon -7566196 true true 158 81 110 129 131 143 158 109 165 110 Polygon -7566196 true true 216 83 267 123 248 143 215 107 Polygon -7566196 true true 167 163 145 234 183 234 183 163 Polygon -7566196 true true 195 163 195 233 227 233 206 159 sheep false 15 Rectangle -1 true true 90 75 270 225 Circle -1 true true 15 75 150 Rectangle -16777216 true false 81 225 134 286 Rectangle -16777216 true false 180 225 238 285 Circle -16777216 true false 1 88 92 spacecraft true 0 Polygon -7566196 true true 150 0 180 135 255 255 225 240 150 180 75 240 45 255 120 135 thin-arrow true 0 Polygon -7566196 true true 150 0 0 150 120 150 120 293 180 293 180 150 300 150 truck-down false 0 Polygon -7566196 true true 225 30 225 270 120 270 105 210 60 180 45 30 105 60 105 30 Polygon -8716033 true false 195 75 195 120 240 120 240 75 Polygon -8716033 true false 195 225 195 180 240 180 240 225 truck-left false 0 Polygon -7566196 true true 120 135 225 135 225 210 75 210 75 165 105 165 Polygon -8716033 true false 90 210 105 225 120 210 Polygon -8716033 true false 180 210 195 225 210 210 truck-right false 0 Polygon -7566196 true true 180 135 75 135 75 210 225 210 225 165 195 165 Polygon -8716033 true false 210 210 195 225 180 210 Polygon -8716033 true false 120 210 105 225 90 210 turtle true 0 Polygon -7566196 true true 138 75 162 75 165 105 225 105 225 142 195 135 195 187 225 195 225 225 195 217 195 202 105 202 105 217 75 225 75 195 105 187 105 135 75 142 75 105 135 105 wolf false 0 Rectangle -7566196 true true 15 105 105 165 Rectangle -7566196 true true 45 90 105 105 Polygon -7566196 true true 60 90 83 44 104 90 Polygon -16777216 true false 67 90 82 59 97 89 Rectangle -1 true false 48 93 59 105 Rectangle -16777216 true false 51 96 55 101 Rectangle -16777216 true false 0 121 15 135 Rectangle -16777216 true false 15 136 60 151 Polygon -1 true false 15 136 23 149 31 136 Polygon -1 true false 30 151 37 136 43 151 Rectangle -7566196 true true 105 120 263 195 Rectangle -7566196 true true 108 195 259 201 Rectangle -7566196 true true 114 201 252 210 Rectangle -7566196 true true 120 210 243 214 Rectangle -7566196 true true 115 114 255 120 Rectangle -7566196 true true 128 108 248 114 Rectangle -7566196 true true 150 105 225 108 Rectangle -7566196 true true 132 214 155 270 Rectangle -7566196 true true 110 260 132 270 Rectangle -7566196 true true 210 214 232 270 Rectangle -7566196 true true 189 260 210 270 Line -7566196 true 263 127 281 155 Line -7566196 true 281 155 281 192 wolf-left false 3 Polygon -6524078 true true 117 97 91 74 66 74 60 85 36 85 38 92 44 97 62 97 81 117 84 134 92 147 109 152 136 144 174 144 174 103 143 103 134 97 Polygon -6524078 true true 87 80 79 55 76 79 Polygon -6524078 true true 81 75 70 58 73 82 Polygon -6524078 true true 99 131 76 152 76 163 96 182 104 182 109 173 102 167 99 173 87 159 104 140 Polygon -6524078 true true 107 138 107 186 98 190 99 196 112 196 115 190 Polygon -6524078 true true 116 140 114 189 105 137 Rectangle -6524078 true true 109 150 114 192 Rectangle -6524078 true true 111 143 116 191 Polygon -6524078 true true 168 106 184 98 205 98 218 115 218 137 186 164 196 176 195 194 178 195 178 183 188 183 169 164 173 144 Polygon -6524078 true true 207 140 200 163 206 175 207 192 193 189 192 177 198 176 185 150 Polygon -6524078 true true 214 134 203 168 192 148 Polygon -6524078 true true 204 151 203 176 193 148 Polygon -6524078 true true 207 103 221 98 236 101 243 115 243 128 256 142 239 143 233 133 225 115 214 114 wolf-right false 3 Polygon -6524078 true true 170 127 200 93 231 93 237 103 262 103 261 113 253 119 231 119 215 143 213 160 208 173 189 187 169 190 154 190 126 180 106 171 72 171 73 126 122 126 144 123 159 123 Polygon -6524078 true true 201 99 214 69 215 99 Polygon -6524078 true true 207 98 223 71 220 101 Polygon -6524078 true true 184 172 189 234 203 238 203 246 187 247 180 239 171 180 Polygon -6524078 true true 197 174 204 220 218 224 219 234 201 232 195 225 179 179 Polygon -6524078 true true 78 167 95 187 95 208 79 220 92 234 98 235 100 249 81 246 76 241 61 212 65 195 52 170 45 150 44 128 55 121 69 121 81 135 Polygon -6524078 true true 48 143 58 141 Polygon -6524078 true true 46 136 68 137 Polygon -6524078 true true 45 129 35 142 37 159 53 192 47 210 62 238 80 237 Line -16777216 false 74 237 59 213 Line -16777216 false 59 213 59 212 Line -16777216 false 58 211 67 192 Polygon -6524078 true true 38 138 66 149 Polygon -6524078 true true 46 128 33 120 21 118 11 123 3 138 5 160 13 178 9 192 0 199 20 196 25 179 24 161 25 148 45 140 Polygon -6524078 true true 67 122 96 126 63 144 @#$#@#$#@ NetLogo 2.0.2 @#$#@#$#@ @#$#@#$#@ @#$#@#$#@