Added new features to Blagominer: support for HTTPS connections and servers requiring custom headers

Quick project info

This software is a "mining" software for a family of cryptocurrencies based on so-called Proof-of-Capacity. The best information about that concept is available in info materials from the BURST coin. You can learn some technical bits from resources on their home page, for example this whitepaper.

Repository

GitHub: https://github.com/quetzalcoatl/blagominer/

Sidenote: This is somewhat tricky. You may notice that it has been forked from andzno1's repository. That's true, but as far as I know, andzno1 doesn't continue this project, so I took it over and forked my own from it. Actually, you can find many more old copies of this project, because it moved from person to person several times. Often by copying, not forking, so GitHub does not show the whole derivation chain properly.

Those historical ones that I've found are, in reverse order:

• andzno1
• JohnnyFFM
• Quibus
• Blagodarenko

Utopian (re)Submission note

This is a resubmission of previous post under the same title. The previous one was rejected due to some misunderstanding. I discussed it with Knowledges and got green light to shorten the post and resubmit.

New Features

This submission is related to my work on this project between Jun 3, 2019 - Jun 2, 2019, with later socket leak bugfixes from Jun 7, 2019, important stability update.

At this point of time I considered it to be a bit too early and untested, so I have not published a "release" yet, however you may see those improvements included in v1.200001.0 released a bit later

1. Adding custom pool-specific headers and parameters to DL submission

Feature branch for this change is feature/custom-headers.

My work on this project began with the idea of making the Blagominer easier to use by small causal miners. While setting up BURST mining is pretty straightforward, setting up BHD mining often requires use of a so-called proxy that injects some custom headers, like X-Account which identifies miner's account in the pool website. An example of such BHD mining pool is The Foxy Pool. Blagominer could not be connected directly to this pool due to X-Account http header required by the pool, as otherwise the pool could not track and transfer earnings properly. Typical installation targetting Foxy Pool so-called Foxy Proxy as a mediating node.

These changes turned out to be pretty simple, so most of the work was actually contained in a single commit:

• added new configuration section for configuring headers
• a bit of code to prebuild and cache a part of the HTTP query related to headers

In a similar way, I also included support for custom http header for BURST connections, and added support for custom http query parameters fo BURST/BHD sections.

2. Support for HTTPS connections

Feature branch for this change is feature/https.

Simply speaking, the Blagominer lacked any HTTPS support, so it could not connect to any pool using HTTPS. Also if the pool used services like Cloudflare, these could attempted to force a redirect to a HTTPS connection, causing Blagominer to fail.

To solve this, I added HTTPS support to the core 3 networking operations performed by the miner, namely: poll(), send_i(), and confirm_i(). Later it turned out that those are actually two operations, known better as getMiningInfo (poll) and submitNonce (send_i, confirm_i).

Changes for this part were far more extensive and spanned about 15 commits. To make a long story short, let me just say that I:

• added a UseHTTPS per-coin switch to the miner.conf file
• refactored actual transport operations away from poll(), send_i() and confirm_i()
• implemented separate HTTPS implementations with help of curl, preserving existing logic as much as it possible
• duplicated sessions queue that held pending sockets, and adapted the copy to hold pending curl handles

After those changes, I began testing Blagominer on Foxy Pool without proxy software, and early tests proved it runs fine and cooperates with the pool properly.

3. Cleaning up socket leaks.

Feature branch: get-rid-of-socket-leaks

Longer HTTPS tests exposed some connection-managing issues. A pool of dead sockets was piling up (detected with ProcessHacker2). After a few 8-hour test runs, each gathered tens or hundreds of stale sockets. On Jun 7, 2019 I managed to fix that.

It turned out that almost all of original poll(), send_i() and confirm_i() had handle leaks. When I implemented HTTPS support, I deliberately preserved old logic to ensure I don't break anything accidentally. However, while on the main flow these routines cleaned up everything properly, in many error handling branches it did not, and now both HTTP and HTTPS code leaked.

A series of 5 commits takes care of them well. After that I performed another set of 8-hrs test runs later and number of open connections stopped growing.

Most of those patches are just adding proper cleanup on specific error code branch that lacked it, but in some places the number of missing cleanups was so large, that I leveraged smart-pointers to guarantee a RAII-style cleanup. An example of that is seen for example here where unique_ptr is exploited for this purpose by using a lambda with closesocket as its Deleter.

While bugfixes usually doesn't qualify, these leaks actually originate from "not my code" :) and I consider them an important "icing on the cake", as these leaks were present there for a long time and these patches significantly reduce socket/port consumption and improve stability of the miner.

GitHub Account

https://github.com/quetzalcoatl/

Fine print section

1. The BURST image used in this article comes from wikimedia.