Complete Datasets for Fischer and Carlsen as White

If you read this blog regularly or saw my work featured at ChessBase, you will be familiar with the tools I have developed to analyze square utilization and occupancy, and represent these as heatmaps. In response to a reader comment, I am providing available for download the complete datasets for both Fischer and Carlsen as White. I may eventually post the corresponding datasets of these two world champions playing Black.

This data is being provided for now with very limited annotation (Select 'Read More' to view); it is up to you, dear reader, to decide how to use it, what points to examine further, and what conclusions you can draw from it. If you download these ZIP packages, you will find the calculated analysis of square utilization and occupancy, as well as the differential data and a subfolder containing heatmap representations for each of the White and Black pieces (all 12 piece types).

Remember, a scientific approach to chess only means that you are willing to test your own ideas about the game in a systematic way, hoping to improve your understanding and thus your performance. Hopefully, the tools and datasets I have made available will assist you in asking scientific questions about chess. In the future, I may post the insights I have gleaned from analyzing this and similar datasets. This future may have to wait some time however, as I have been rather busy lately with my professional commitments!

Select 'Read More' to see details regarding methodology as well as a few initial insights. Please feel free to share your own insights from this dataset, or your critical comments regarding this method! Stay tuned for more science on the squares!

Finding Difference Makers

Using JavaScript tools I recently created, you can determine the square utilization or occupancy for any particular piece, for any given square, across all moves and all games in a PGN database. One potentially useful application is to apply these tools separately to the wins and losses (1-0 versus 0-1) in the database of your choice. As featured in a recent ChessBase article, I have done this to find which piece movements or placements are featured more often in wins versus losses. These are potential difference makers in a chess game.

In this post, I will describe in more detail how I performed these calculations. Although this can be done by hand, I created an Excel spreadsheet to facilitate processing of the data you can get from using the aforementioned tools. This spreadsheet is available for download; select 'Read More' to see the rest of this article to find instructions on how use the excel file, as well tips on how to represent the results using the heatmap tool. Stay tuned for more differential data and analysis in the coming weeks.

As featured on ChessBase

It was very exciting to see this morning that the editors at ChessBase have published the article I wrote that describes the chess tools I have recently posted about. For those who haven't seen it, I encourage you to give it a look.

Some of you may be viewing this blog for the first time due to the ChessBase article. To all of you, I say welcome! Although my updates to this blog are not that frequent (once a week at best), if you found the article interesting you should subscribe or keep your eye on this space. Here is some of what you can come to expect in the near future from Science on the Squares:

• The excel file 'tool' I use for normalizing the data I collect and finding the differential between two data sets, available for download along with instructions
• More information on how I created the heatmaps, and why I used particular settings
• A post concerning what I think the proper use of these tools are. This will expand upon my thoughts from a previous post, in which I argued for a scientific approach to chess.
• Responses to the comments by readers of the ChessBase article.
• More data and analysis of players and openings! In addition to the Sveshnikov Sicilian, I have already looked at the Winawer French, the Ruy Lopez Exchange Variation and Breyer Variation, and the Smith Morra Gambit.

Thanks again to all my readers, and a special thanks to the editors at ChessBase!

Chessboard Heatmap and Updates

In a previous post, I introduced a suite of Javascript tools that can be used to reformat PGN files and determine square utilization (traffic) and square occupancy (parking) for different pieces. Since that post, I have rewritten the code and improved the functionality of each of these tools. I have used these tools to collect data on square utilization or occupancy, which I then process further in an excel file (which I will make available shortly) and generate heatmaps using Plotly. However, Plotly was cumbersome to use, having to reformat the heatmap for each data set.

Here, I am introducing another Javascript tool that can be used to take square utilization or occupancy data and generate customizable heatmaps specifically for chess. This can be found at the following address: http://djcamenares.x10.mx/chess/heatmap.shtml

In the rest of this post, I describe how to use the features on the heatmap program, as well as detailing other updates to the site. Select 'Read More' to view the rest of the article. Please feel free to share your comments regarding this tool, especially if you used it to generate interesting insights!

Tools for Statistics of the Squares

Ever wonder which squares see the most traffic? Or which squares have pieces 'parked' on them for the longest periods of a game? Now, you can answer these questions easily using a series of JavaScript powered tools I have developed. These are based upon the LT-PGN JavaScript viewer, and calls upon that code (which I did not write) for certain functions. In fact, it directly calls upon the LT-PGN PGN2FEN tool, although my adaptation can handle PGN files with multiple games. 

I was inspired to develop these tools after reading some of the chess-visualization articles posted on the ChessBase website, namely, the analysis of square utilization by Seth Kadish earlier last year (analysis which you can also find at Seth's blog). I really liked the approach, and although the source of the game PGNs were made clear, I wasn't sure how he extracted the data. Also, I thought there was potential to observe more than just square utilization, or 'traffic'.

In order to carry out this analysis, I created three separate JavaScript tools. They aren't shining examples of efficient coding, but they get the job done. They are as follows:

Reformatting PGN Text
http://djcamenares.x10.mx/chess/pgnreform.shtml
Although PGN files from 365chess.com can be used directly in the downstream applications, the chess software I use (HIARCS) places line breaks within the game score. This tool will remove them, leaving all other features of the PGN intact.

Move Counting (Determining Square 'Traffic')
http://djcamenares.x10.mx/chess/traffic.shtml
This tool takes a PGN file, with single or multiple games, and can determine the square utilization or traffic of the White player, Black player, or both players. It also reformats the PGN so as to remove the header tags.

Batch Conversion of PGN to FEN, Counting Square Occupancy ('Parking')
http://djcamenares.x10.mx/chess/parking.shtml
This tool takes a PGN file, with single or multiple games, and does several things. First, it removes header tags from the PGN. Then, it converts the PGN first to FEN, then to an expanded version of FEN in which each square, filled or empty, is declared. Finally, the occupancy, or parking, of different pieces (selected by user) on each square is reported.

To see some example results of these tools, which expands upon the aforementioned work by Kadish, please select 'Read More' below. 

Practical Chances and a Romantic Opening Shootout

In keeping with the new focus of this blog, I decided to carry out a small experiment regarding opening theory. As a way of judging the merits of several romantic opening lines, such as the Pierce Gambit, Frankenstein-Dracula variation, and Halloween Gambit, I have performed a series of Engine-Engine matches using HIARCS (I use a Mac).

In these matches, I varied the time allotted to the engines for making moves, in order to determine in a quantitative fashion if a particular line offers practical chances and pressure, but can be defused with accurate defense. In the complete article below, I explain my motivation, methodology, and my results. This is a preliminary analysis, and future posts will continue this work, for example by looking at engine evaluations and comparing them with the results.

Still, a few interesting results were apparent. In particular, I was interested in several lines in the Pierce gambit, which starts with 1.e4 e5 2.Nc3 Nc6 3.f4 exf4 4.Nf3 g5 5.d4 g4 6.Bc4 gxf3 7.O-O Nxd4! (Strongest test against the Pierce) 8.Bxf4 (the Knight is immune due to threats of Qg5 followed by Bc5 or Qg2) Bc5 9.Bxf7?! Kxf7 10.Be3

According to my analysis, the line with 9.Bxf7 Kxf7 10.Be3 is inferior for White, as others have suggested in the past. Notably, Ian Simpson at his excellent blog and site, The Gambiteers Guild, suggests that this might be playable for White, if it wasn't for 10...Qf6 11.Nd5 Qe5 12.Rxf3, when White's compensation falls short. In fact, 12.c3 instead should lead to at least an even game, with some pressure and practical chances for White. Some of the computer lines that I generated from this line are both amusing and instructive.

Select read more to see the complete article, with results, discussion, and my methodology. Think I missed something, or have an opening you'd like analyzed this way? Please feel free to share in the comments below.

Hello to a New Journal, Farewell to a Friend

The Journal of Chess Research is a new scholarly or academic journal that will feature scientific articles and studies centered around chess. In some ways, this project is a more formalized and serious version of this blog itself (with a much more impressive board of editors and potential authors). The journal itself was featured in another chess publication, Chess Life (official magazine of the USCF) in a recent issue. You can also read more about this project at their website or at Susan Polgar’s excellent blog. 

            Speaking of excellent blogs, one of my favorite chess authors and friend, Michael Goeller, is hanging up his hat with regards to the Kenilworthian. You can read about his decision to do so, and a reflection on his amazing blogging journey, at his final post, The End of Chess Blogging. His articles were always interesting, well written, well researched, and very enjoyable to read. For at least one reader (myself), his writing will be sorely missed.

            I may be reaching my own end of chess blogging; work commitments to both teaching and research continue to prevent me from making consistent posts. I will be sacrificing time that could be used for Science on the Squares in order to revive my other blog, Just Me and Eubacteria, which is more in-line with my professional interests. Towards this end, I have removed some of the content from this blog, scrubbing out broken links and images where I can. I may occasionally post a game or analysis here or there, but regretfully these posts will be few and far in-between.  

A Scientific Approach to Chess

For a blog about Chess and Science, you might expect more articles about the intersection between the two. Indeed, I have occasionally written here about research articles that have involved chess (such as studying the physiology of players during a game, the Einstellung effect), even including my own efforts in exploring mutual information in chess. Usually, chess is used as a vehicle in science to study physiology, memory, or decision making in general. However, I think it is worth exploring a truly scientific approach to Chess.

In its essence, Science is about gaining useful knowledge in a systematic way to solve problems. Chess players engage in a similar activity all the time, if even subconsciously, studying games and reading literature to build a model of the game in their mind that can be applied to making decisions at the board. Likewise, scientific knowledge about the natural world has informed incredible advances in technology for a wide range of industries.

Below I suggest different ways in which the scientific method can be applied to different aspects of chess. I encourage any interested readers to take up the challenge of performing chess research, following the principles featured in the rest of the article. I intend on making this blog a vehicle for such research, and welcome submissions of original research to this blog (or at least posts that link to your analysis). Perhaps Science on the Squares can become the first real scientific journal of chessology (or chessonomics?)

Do you think that such a method will prove useful and yield insights? Is it too slow and laborious? Have you made any discoveries in Chessology? Please share your thoughts and comments below, or contact me directly. 

Evolutionary Biology and Education

The internet is perennially abound with discussion over evolutionary biology. It is a topic that surfaces repeatedly in the news, either in the form scientific dispatch describing a new species or in political news concerning the friction between science and religion. The latest iteration of the latter debate concerns recent remarks by Florida Senator Marco Rubio when answering a question about the age of the earth. The Senators somewhat ambiguous and confused (but understandable and unsurprising) answer has found both critics and apologists. He eventually qualified his answer, and I think made a good point that I will echo later in the post: the age of the earth has nothing to do with his tasks in the Senate. Even this point has been criticized, but I think it rings true in a subtle way).  Many of these debates, however, usually filter back to the opposition between creationism and evolution (as well as geology, embryology, or any other science that touches on the same points).

(Taken from Joyfully Battle Worn's blog without permission. Interesting post, you should definitely check it out. Echos some of the sentiments I expound here, although approached from a more religious perspective.)

The astute reader may be queuing up their groans and moans (or cheers, if they are into this debate). "Will this blog also devolve into a debate over evolution?" While I thought it would be appropriate to weigh in on the subject (after all, this blog is partly devoted to science, and biology in particular), you need not fear that these pages will transform into another blog taking on evolutionary biology. After all, there are many fine sites that primarily focus on this topic. I am especially familiar (and fond of) some of the more popular ones maintained by other scientists, such as PZ Meyer's Pharyngula. There is even a blog, aptly named EvolutionBlog, that focuses on both evolution and chess!

What then, does Science on the Squares have to offer to the evolution blogosphere? I am trained in molecular biology, not evolutionary biology (although I am well-read on the subject), and thus I will leave the discussion of the finer points of this science to others. However, I would like to add a voice to the debate over the proper placement of evolution in the classroom.

I've beaten about the bush long enough; my point is that neither creationist science, intelligent design, or evolutionary biology (as I understand it to be currently taught) belongs in the science classroom. No, you don't need to adjust your screen or check your glasses: I did indeed call for the partial removal or reform of evolutionary biology from the Science curriculum. 

Now that I have your interest, please select 'Read More' to read this article in its entirety. I'll give you a preview: the evolutionary explanation of the origin of species is more history than it is predictive science, and thus is not critical to the education (or future competitiveness) of our youth.

The position I've taken is one that is sure to stir up controversy, and will probably have the somewhat unique distinction of angering proponents of both evolution and creationism. Please feel free to add to the discussion by leaving a comment below, but try to keep it civil!

The Science of Chess: Moving to the (heart) beat

Recently a team of researchers published an article (PDF format) in which they monitored the heart rate of Chess players during a game (Aptly titled The tell-tale heart: heart rate fluctuations index objective and subjective events during a game of chess. Published in Frontiers in Human Neuroscience, 2012, Vol 6, No 273). 

The same authors of that study have summarized their work for chessbase news in a report also aptly titled: "Psychology: The Heart of Chess". Naturally, I was interested, as I have posted before about the intersection of science of chess and I check PubMed occasionally for such work (although I must have missed this particular study).

Picture reposted (with modifications) from Chessbase. I do not own the rights to the above picture (or for that matter, any of the pictures used in this post).

The article is interesting and thought provoking. There have been many studies that examined some psychological or mental aspect of the game, but there are much fewer (to my knowledge) that studied the physiological manifestations of a chess player's mind. As the article reminds us, Chess is fertile ground for studying all of these phenomenon. 

What did you think of the article? Select 'Read More' to see my full opinion. Please share your thoughts in the comment box below!

Mutual information in Chess

In a previous post, I introduced the topic of mutual information. This is a statistical analysis technique that can be used to determine if there are a connection between two variables.

Mutual information finds many applications, including analysis of biological sequences (this is the use I am most familiar with). Last time, I questioned whiter or not this technique can be applied to chess in any meaningful way. I think the answer is a partial yes, and here I will share with you my initial exploration of this idea.

Introduction to Mutual Information

Despite the vague sounding term, the concept behind Mutual Information is a rather simple one. It is essentially a measurement of the dependence between two random variables. Huh? Put simply, it describes a sort of correlation between the outcomes for two different varying quantities.

Below, I try to introduce the concept and some uses of Mutual information. Since my experience in using this statistical technique has been one focused on bio-informatic analysis, most of the examples and uses I describe relate to biology and sequence analysis. There are a multitude of other uses! To read the rest of this article, select the 'Read More' Link below.

What ways have you used mutual information, or what problems do you think it can be applied to? Please share below by leaving a comment.

The image above (and some below) is from Wikipedia (I claim no rights to them)

Occam's Razor in Science and Chess

Somewhere along their education, most scientists learn about Occam's razor. This principal, attributed to the 14th century logician William of Ockham, is usually stated as the preference for a simpler theory over a more complex one (as long as both are supported by the facts). When formulated this way, it might also be called the law of parsimony or economy, or just the rule of simplicity.

Occam's razor does not only apply to the hard sciences. This rule has been applied, sometimes in a modified form, to many fields. Even chess players may utilize Occam's razor. After all, while scientists use experimentation to falsify competing hypotheses, chess players engage in a similar activity by evaluating competing moves through analysis (often with computer help). Just as two different hypotheses may explain the facts, two candidate moves may appear (at first blush) to be playable. In either case, experimentation or analysis is used to find the correct choice between the alternatives.

I would suspect, however, that many scientists  do not appreciate the rationale behind this razor, as well as it limitations. They may be in danger of over estimating the power of this principle. In fact, I'd wager that chess players are more aware of the proper use of Occam's razor. Some of the original justifications for the razor where aesthetic (simpler theories and more elegant moves must be better), but this rationale is quite simply irrational. There is no good reason to believe a priori that either the inner workings of the cell or the strategy on a chess board must be simple. Even if many successful theories rely on simplicity, this does not preclude a phenomenon (or a position) being studied that requires a complex explanation.

Select 'Read More' to see the complete article, in which I discuss the justifications for Occam's razor using examples, and exceptions, from the chess world. The law of simplicity is more of a guideline, a way of prioritizing experiments and guarding against circumstantial theories. 

As Sherlock Holmes might say, understanding Occam's Razor is "Simplicity itself!"

The Science of Chess: Fernand Gobet and the Einstellung effect

Since this blog is about Chess and Science, it is only natural to discuss the nexus between the two: scientific studies that examine some aspect of Chess. The game of chess is used by some social science, psychology and neuroscience researchers as a model or tool to examine memory, expertise, decision making skills, or some other process that two combatants engage in over the chessboard.

A few chess players may be already familiar with an example of this type of work, perhaps the highly cited studies by Adriaan de Groot. In some of the studies conducted by de Groot, chess players of different skill levels were tasked with position recall and were told to analyze a position while thinking out loud. This has led to conclusions about the way expert players organize their memory, and how they search and evaluate a position. (I'll probably revisit de Groot in a future post)

A contemporary researcher in this field is Fernand Gobet, a professor at Brunel University, West London. (See his professional homepage). Gobet has dual qualification to study chessplayers, since he is both a cognitive psychologist and an international master (a qualification not unique to the researchers in this field). He has published a number of interesting articles on a variety of aspects of the game, ranging from memory, visualization skills of chess players, and even gender differences between players. Of particular note, Dr. Gobet has attempted to simulate certain aspects of expert memory and skill using commuter models.

Below, I'll give you my reaction to a paper by Dr. Gobet that deals with the interesting Einstellung effect. While I am scientist, cognitive psychology is not my field, so the experts will have to forgive me if I do not assess the work correctly. Also, I provide below only a brief sketch of the work; I might someday blog about the individual topics in more detail, as I learn more about them.

I'd love to hear from you, dear reader, if you have some insight onto to the work of Dr. Gobet or others. In particular, please feel free to let me know if I have gotten something wrong. You can do so by leaving a comment.

Molecular Biology Primer: Obtaining and Interpreting Data

As mentioned in the previous post in this series, molecular biologists are primarily concerned with studying the expression, structure, and function of proteins in the cell (sometimes this necessitates the study of the genetic material which encodes the protein, particular when the proteins expression is being questioned). 

How do scientists go about answering these questions? What are their tools of the trade, how are they used, and what are their limitations? Here I explore some of the broad limitations common to many techniques used in molecular biology. If you'd like to learn more about some of the more common techniques, I have provided at the end of the article short descriptions and ample links for more information.

I'd also like to hear from you! What experimental limitations have I omitted or underestimated? If you know of some cool new techniques, please help me grow the list I have compiled by adding a comment.

Molecular Biology Primer: The Central Dogma

The science of molecular biology is found in the latest cancer treatment, behind the development of antiviral and antibiotic drugs, and even in today's superhero movies (where the hero's DNA is altered through a lab experiment, spider bite, etc). But just what the heck is molecular biology? Do you need a degree in biology or chemistry to understand and appreciate what occurs on the smallest scale in every living organism?

This is part of a primer written for readers with very little knowledge of biology. I have compiled this brief explanation of biological information, as well as some resources for further studies, so that such readers will still be able to access and digest any future posts with commentary on scientific topics.

In the rest of the post, I hope to accomplish a few things. First, through a musical analogy of the journey from sheet music to sound, you will learn the so called 'central dogma' of molecular biology, the flow of information which is at the very heart of life. This will lead into an explanation of how this information is ultimately expressed in the form of the function of proteins, how this expression is controlled, and some of the consequences of protein function.

My depiction of the central dogma and the flow of information from genetic material to protein function. For the aficionado, I attempted to make this as accurate as possible: the mRNA sequence translates to the protein shown, and the protein is folded in order to shield the hydrophobic residues A, V, and L.

In a future post, I will describe in more detail some of the methods researchers use to study molecular biology. As a teaser, I highlight here some typical questions posed by molecular biologists. (Note: these questions are probably more typical of academic scientists. This is another product of that axiom, 'write what you know'). Finally, I will provide some resources for further study (in particular, check out of the videos from the Dolan DNA Learning Center. They are really cool!).

Square One: An Introduction

Chess is everything: art, science, and sport.  -  Anatoly Karpov

For me, chess is at the same time a game, a sport, a science and an art. And perhaps even more than that,. There is someting hard to explain to those who do not know the game well. One must first learn to play it correctly in order to savor its richness.  -  Bent Larsen

Chess is a unique cognitive nexus, a place where art and science come together in the human mind and are refined and improved by experience.  -  Garry Kasparov

Welcome to Science on the Squares, a blog about Chess and Science, and everything in between.

I've heard on occasion the cliche "Write what you know". In some ways, it is through this phrase I arrive at this blog. I am an avid chess player (although not particularly strong, only USCF 1666 at time of writing), and I am also currently drawing my doctoral studies in molecular biology to a close. Thus, I am in position to comment on both worlds. As the quotes above (grabbed from http://www.chessquotes.com/topic-chess) suggests, these worlds may intersect in some ways.