17

I've been hearing this claim being bandied about quite a lot and I'm wondering if there's any substance to it. Apparently the details are in a certain thread on Talkchess, but it's not viewable to non-logged in readers, and it won't let me register. There are no details about this on the Houdini Wikipedia page either.

Can anyone give a summary of the allegations and the evidence for (and against) it?

Allure
  • 25,298
  • 1
  • 65
  • 143

5 Answers5

13

Yes.

The search, evaluation, etc., are almost identical to Stockfish, with the notable exception of being rewritten in Dutch. A number of Stockfish developers, including former maintainers, have noted as such. To hide its origins, code was included that multiplied the node count by 8/7. Exploiting modular arithmetic tricks and archived Houdini games, this exact code was shown to be present in the release version of Houdini. The alleged performance improvements over Stockfish go away after removing this trick.

The leaked source code was compiled by Ed Schroder and found to have a 1:1 correspondence with Houdini in a vast number of test positions, while others, like Simon Guenther, have found that the leaked code behaves identically in the start position even up to high depths.

There were changes, but Mark Lefler, developer of Komodo, noted that a number ofaa changes were simply adding features reverse engineered from Komodo. In fact, in one file, evaluatie.cpp, you can spot the naming conventions used by the Hex-Rays decompiler.

Other features were unique, but minor. For example, the licensing system was not from Stockfish. However, the code came from TurboPack and was translated into C++, so it's not clear that that was original work, either.

Adjusting for the time of release, the version of Stockfish on GitHub at the time of Houdini 5's release was more or less equal in strength to Houdini 5. Later, when Houdini 6 was released, this was also true. (Source: nextchessmove's elo data + CCRL data)

In other words, Houdini did not notably build upon Stockfish in any meaningful way. The main reason it looks different is because of the Dutch translation and the fact that Stockfish has evolved significantly since Houdart last touched the Stockfish codebase.

Aaron Williams
  • 156
  • 3
  • Is there an explanation for why Houdini looked so different then? For example, I notice a lot of its games which end up drawn which it's playing White, its eval drops to slightly below zero before going back up to 0.00, which Stockfish doesn't do. Houdini was also the strongest engine on CCRL for a few months, I think. – Allure Apr 12 '20 at 20:43
  • Stockfish has new official versions out every ~six months. While there's no official cadence, releases come slowly while the Stockfish version on the GitHub repo is constantly updated. At the time Houdini 6 was put on CCRL, Stockfish was at least 35 elo stronger than CCRL's version, based on [NCM](https://nextchessmove.com/dev-builds) data. The source code looks nearly identical to SF's, minus the variable name changes. However, since it was based on an intermediate Stockfish version, incorporated Komodo code, & had small tweaks, it played a bit differently from any official SF releases. – Aaron Williams Apr 12 '20 at 20:52
  • Hmm I remember a time when Houdini was the strongest engine, better even than Stockfish dev - e.g. right around the time when it won TCEC Season 10. How is that possible if it is almost identical to Stockfish? – Allure Apr 12 '20 at 21:50
  • During TCEC Season 10, Stockfish *was* the strongest program. Ironically, precisely because it was so strong, it would calculate that there was danger in many positions and trade down to draws in every game. Right after that season, the Stockfish people decided that playing without some sort of contempt was a terrible idea, and SF has been in every superfinal since. Houdini got in instead of Stockfish because it was configured to trade into draws against weaker engines. Now, Houdini versions <= 4 were at times better than Stockfish. Houdini 1.5, for example, certainly was at time of release. – Aaron Williams Apr 12 '20 at 22:36
  • Is there any evidence the source code is genuine (i.e. not reverse-engineered)? After all if some features were reverse-engineered from Komodo, presumably it would be possible to reverse-engineer Houdini and then pass off the code as its source code. – Allure Apr 13 '20 at 06:25
  • 1
    Houdini at one time, was the world's strongest, it consistently top Stockfish in all ranking lists and TCEC. How would you explain it? – ABCD Apr 13 '20 at 08:13
  • 1
    @SmallChess It did not consistently top Stockfish. Stockfish lost season 10 because it was not configured to punish weaker engines, as Komodo and Houdini were, so it did not promote past Division P. For that season, it earned the name "Drawfish" despite playing the most precise chess. Prior to the rise of Fishtest, Houdini beat Stockfish by building on top of Robbolito. This is well documented. (See Richard Vida's analysis.) After Fishtest, Stockfish's dominance became insurmountable. Rating lists generally don't use the development versions of Stockfish, which could easily be >40 elo stronger – Aaron Williams Apr 13 '20 at 23:33
  • @Allure Yes. Between the comments, the Dutch--matching Houdart's coding style--inclusion of a reference to a GitHub repository that was deleted a long time ago, a number of source code comments, leaked installer generator code, the fact that Houdini 5 was also reverse engineered, etc., it's essentially guaranteed to be legitimate. – Aaron Williams Apr 13 '20 at 23:36
  • @AaronWilliams thanks for answer. Based on konsolas's screenshot below, cucumber says there's a "good deal of original work" in Houdini. Do you know what that original work is? – Allure Apr 14 '20 at 05:00
  • Not a clue. After mjlef (a Komodo developer) commented that there were a number of reverse-engineered Komodo features, it's hard to tell what was borrowed and what wasn't, since Komodo is closed source. – Aaron Williams Apr 14 '20 at 17:32
5

Looks like the answer to the title question is "yes". Stockfish sued Chessbase (a distributor of Houdini) for violation of GPL, and in the settlement, Chessbase concedes that they "used Stockfish in violation of its license in two of their products, Fat Fritz 2 and Houdini 6".

Allure
  • 25,298
  • 1
  • 65
  • 143
4

A user on talkchess.com (by the name of cucumber) claims to have come into possession of Houdini's source code, and has noted a number of similarities between it and Stockfish's source code. In particular, they note that the Makefile contains the phase 'to build Stockfish' and they link a screenshot of a search function which appears to be Stockfish's, but translated to Dutch:

initial post by cucumber

It is unclear if the source code they have is actually Houdini, though a few contributors to the thread allege that the compiled source closely mirrors the behaviour of the actual engine binary.

Additionally, cucumber claims that they obtained the source code illegally. post clarifying how the source was obtained

It seems that all claims here hinge on whether or not cucumber actually has Houdini 6's source code. Since they have stated their intent to not release it publicly, this will be difficult to verify.

konsolas
  • 3,166
  • 15
  • 32
  • I thought it was published? – ABCD Apr 12 '20 at 14:55
  • "I do not intend to publicly release any of Houdart's original code. I also do not intend to share all of the source with anyone privately. Only what seems necessary." - cucumber. – konsolas Apr 12 '20 at 14:58
  • It's possible that it was published at some point in the 38 page (as of now) thread, in which case I must have missed it. – konsolas Apr 12 '20 at 14:59
  • The source code has been published on Github, and compiling that code creates an executable that's basically Houdini 6.As to the legality of obtaining that code, it likely depends on your legal jurisdiction. The repository was publicly accessible on the internet. – HTTP 410 Apr 12 '20 at 17:41
  • 1
    Thanks for answer, but can you link to a screenshot instead of that thread since it's not accessible without logging in? – Allure Apr 12 '20 at 20:44
4

The alleged source code might have been published online. I took a quick look into it, it looked like a valid strong chess engine. It was not a clone of Stockfish in the sense that the codebase was quite different, certainly not a 10 minutes copy-and-paste. It would've taken even the best talent months (or years?) of hard work to develop it, so it doesn't look like a prank. It certainly looks like the Houdini source code. I can't find anything in the code that says otherwise.

My personal feeling is the Houdini author must have studied Stockfish's source code very carefully. But ... honestly who wouldn't? Komodo is famous for tracing every single bit in Stockfish and apply/improve the ideas in their own engine. I believe everyone who is serious in chess engine programming would treat the Stockfish source code like a bible.

I feel Houdini was unsurprisingly sourcing information/ideas etc from Stockfish and Robolitto, but it was not a clone more like a genuine efforts to improve old ideas and develop new methods.

The definition for cloning is confusing. Chess engine authors believe a "clone" happens if an idea (or some code) copied from Stockfish. The general public would think a "clone" is a direct copy-and-paste from Stockfish (maybe just change the program name in UCI?).

We need to work out the definition.

PS: the code looked like a stolen product:

enter image description here

ABCD
  • 22,419
  • 2
  • 43
  • 82
  • 4
    Apparently Houdini's nodes per second is inflated by 7/8, which can be verified by examining its actual speed. In other words, Houdini never outputs a nodes per second which is 7 modulo 8 (and it indeed never does). Any idea how this can be explained other than as dishonest reporting? – Allure Apr 14 '20 at 05:02
4

I saw the code and there is no doubt that code was stolen form Stockfish Here's a small snippet that I saved. I no longer have the code since I found out it also includes code stolen from komodo ( reversed engineer). There are many many examples just like this,

The one of the right was from Houdini, the one I the left is from the Stockfish repository from a few years ago, not current dev Stockfish - commit begins with in "...cd8dc0a" https://github.com/official-stockfish/Stockfish/pull/706/commits/cd8dc0a1016cefe332f41fd7645b4b8462b1e466

I was able to build an exact clone of Houndin 6.03 from the entire source that is on the right. This is just one example - that are many snippets just like this one. It provided exact node count and pv when one performed "go depth xx" on any position with Houdini 6.03.
enter image description here

MikeB
  • 161
  • 1
  • 3