My journey to Mining v2 [Guide]

Dave2kDave2k Member Posts: 46
edited May 2017 in Mining

Hello dear ethereum-community,

with this post, i want to share my journey to mining on a 6 GPU rig. I haven't found a complete guide, that suits my needs, so I figured I could write one myself (or at least try). This is a repost to Version 1!

This post is divided into a section where I'm explaining, what i did to make it work (I.) and a section of things that failed (II.) up to the point of success. At the end i will write down, what i did to increase the hashrate (III.).


I. How did i get it working

First of all, lets see what I bought:

    1 x MSI Z170A GAMING PRO CARBON
    1 x Intel Celeron G3930 2x 2.90GHz BOX
    2 x 8GB Crucial CT2K4G4DFS8213 DDR4-2133 DIMM CL15 (overall 4x 4GB =16GB)
    1 x 120GB Kingston SSD Now V300 2.5" SATA
    6 x XFX 480 RS 8GB
    1 x 1200 Watt Seasonic Prime Modular 80+ Platinum
    6 x PCI-E Express 1x to 16x Extender Riser

Getting started:

After you have assembled everything except the GPUs and their risers. You should be able to boot the system into the BIOS. That is where we start. You have a keyboard and a DVI or HDMI cable in the corresponding output plugged into your motherboard. Prepare a USB thumbdrive with the newest BIOS (Version 1.7) provided by MSI (link). Plug that USB drive into the motherboard. Start the computer and press DEL on your keyboard to enter the BIOS. Once you are seeing the BIOS, you choose Advanced Mode (F7) and M-Flash. This will require the system to reboot into flash mode. The next window you see, will show a folder structure. Find your way to the file you've just put onto the USB thumbdrive. Once you've selected the file it will start flashing the BIOS. This was the very first step

Now we need a working bootable windows 10 pro usb-thumbdrive (at least thats what I use). I recommend rufus for creating such drive. Plug the bootable USB drive into the motherboard and boot it. I assume that the installation is a UEFI installation.

Once W10 is installed, make sure that all necessary drivers (i.e. Chipset-Driver) are installed (not those that are needed for the GPUs!). I've allowed RDP access on my miner to check if everything is alright on my main computer. You can find that option via System, Advanced System Settings, Tab: Remote, Check: "Allow Remote ....". Next setting we want to touch, is the "install drivers automatically"-feature. Go to System, Advanced System Settings, Tab: Hardware, Driver Installation Settings and choose No. If your Windows is updated and configured, we can proceed by downloading the AMD drivers. I am currently using the WHQL-Drivers version 16.10.1 (link). Just download it, don't install it yet. Shutdown your system.

Take one of your GPUs and plug it directly onto the 16x PCIe slot onto your motherboard. Pull the DVI/HDMI cable from the motherboard and plug it into the GPU. Boot your system. It should detect, that the display is now on the GPU rather than the iGPU and you should see the BIOS booting up. Change the setting: Settings, Advanced, Integrated Graphics Configuration, Initiate Graphic Adapter to PEG. Save the settings and reboot.

Once the system is successfully booted, go to your downloads folder where you've put the AMD driver installer and install the driver. I chose to install everything. Your display will flicker during this process. As soon as the installation is complete, try to start "AMD Settings" from the windows menu. If that worked we are good to proceed. (I uninstalled the iGPU via Device Manager at this point, but I'm not sure if thats needed). Reboot your system and enter the BIOS.

In the BIOS go to Settings, Advanced, PCI Subsystem Settings. Here you change PEG0 and PEG1 to Gen1 and enable "Above 4G encoding". Save the settings. The system will try to reboot now, but i shut it down via pressing the power button.

Now its time for the 5 remaining GPUs. Remove the GPU that is sitting on the PCIe 16x slot and plug all risers on the motherboard. Plug the GPUs into the risers and connect everything accordingly. Picture 4 shows my rig. My risers have a SATA to 4-Pin Molex cable, where they are pulling power from. The AMD cards are power hungry and they are not only pulling from the 8-Pin connector, they are also pulling from the PCIe/riser. So i did only connect two risers on one SATA power cable coming from the Power Supply (PSU). If you are connecting more than two risers you can feel the cable getting warm. I do not want that. The GPUs came with a 2x6-Pin to 1x8-Pin adapter. The PSU came with four cables that have two 8-Pin connectors. As you can see in the picture 5, I connected two of the mentioned cables to three GPUs.

Is everything set? Boot your machine, Windows will install the drivers for the remaining cards and you will see flickering for the next 5 minutes. Check the Device Manager if you can see 6 x RX 480. If yes, "high five"!, if not, let me know, I may have got an idea.

The next step is to download mining-software, I am using claymore's miner v9.1 at the moment. Once downloaded and extracted, go to the folder containing EthDcrMiner64.exe and search for the start.bat. Edit the file to your preferences. This is what my start.bat looks like:

setx GPU_FORCE_64BIT_PTR 0
setx GPU_MAX_HEAP_SIZE 100
setx GPU_USE_SYNC_OBJECTS 1
setx GPU_MAX_ALLOC_PERCENT 100
setx GPU_SINGLE_ALLOC_PERCENT 100
EthDcrMiner64.exe -epool eu2.ethermine.org:4444 -ewal 0xfD4d1B9ec7C75Af47787d3449d6e1f85c88B3eF1.Miner01 -epsw x -asm 0

Double click start.bat and the machine should start working. Congratz, you are mining Ethereum! First thing I did was to open "AMD Settings" and reduced the power limit. Go to Gaming -> Global Settings and choose the first RX 480 Wattman. Scroll down until you see power limit and set it to -15%. Do that for all GPUs. Most systems should be stable at -15%. You can fine-tune that to your liking. I will explain more on that topic in section III.


II. What failed and problems I've encountered

This section is ment to be like a "PSA" or "TroubleShooting"-Section.

Which GPU is which?

First of all, GPU0 as shown in the Miner-Console, is not necessarily the card on the most right, or the card on the first PCIe-Slot. To determine which GPUx shown in the Miner-Console is which in reality, I disabled mining on them. You will see the fan stop spinning after a few seconds. I've then created a list with the shown name and their corresponding slot on my rig.

For example: GPU3: 6 P6 2100 -20. GPU3 is shown in the console, 6 is the corresponding number in WattTool (I explain later how to use WattTool to overclock), P6 is the self-given name of the slot on the rig. 2100 is the Mhz of the RAM and -20 is the PowerLimit.

If you press a number inside the Miner-Console, the GPU with the corresponding number will stop mining. As soon as you do that, the "Activity" in WattTool will reduce to ~0% and shortly after that, the fan will stop spinning. Thats how I determined which card is which!

Out of the 6 GPU's I've bought 2 were damaged. And also the first motherboard I've bought was damaged. I had to RMA them. How did I find out which GPU was broken: look for the log in the Claymore Miner-Directory. The very last lines will contain a "Hang-Message" (see below). The mainboard was totally non-functional, so that was easy to guess. It has debug-LEDs, none of them were on, even though the computer didn't boot. I was quite upset to have so many malfunctioning "new" computer-parts.

16 Gb RAM?

Since I had to RMA alot of the stuff, I haven't had all parts together when I started mining. I started with 4 GPUs. So I set everything up accordingly and it worked. As soon as I've gotten the remaining GPUs I've added them to the rig and booted. BSOD right after Windows was trying to install the drivers for the new cards (“ATTEMPTED_EXECUTE_OF_NOEXECUTE_MEMORY"). I tried some things, but nothing solved the Problem, then I've found a recommendation on some forum, that I should install more than 8GB of RAM. I did so. Unfortunately it didn't work. I got it working with the suggested method, which i described in part I. Basicly, install one card on PCIex16 and then plug all other cards in right after. Not one by one. Since it worked, I didn't research much further to explain "WHY", but im guessing Windows is allocating resources and there is no more space for other devices to claim the same amount, but don't trust me on that. ;-)

Remote Desktop Protocol and Hashrate?

I thought I had another broken GPU, because the hashrate of that card was always bouncing. It turned out, when i was checking my miner with the remote console from another pc, "http://<ip-of-miner>:3333" the hashrate was fine and stable. So I assumed, that it had to do something with the RDP-Session on the miner.

There are different RAMs on RX480

There are, as far as I know, three different RAMs installed on RX480s. Which RAM-Manufactor has built the RAM for your card is distributor specific! My cards have Samsung RAM. Other cards can have Elpida and Hynix RAM.


III. Advanced

In this section, I want to cover "advanced" things beyond the installation and troubleshooting.

Overclock / Tuning:

To increase my hashrate I eventually decided to flash the Bios of my cards and optimize their settings. The RX 480 cards have two BIOSs, which are accessable via a small switch near the power pins of each card (see the following picture).

Picture 1

The switch position in the picture is supposed to be the "silent"-position. It is towards the Display-Outputs.

So I shutdown my miner and switched all GPUs to the "silent"-position and rebooted it, then I backed up the BIOS of my cards. I used GPU-Z to do that.

Since I am always sceptical when it comes to downloading stuff from boards and video descriptions I didn't want to download the BIOS someone had prepared, I wanted to modify it myself. That is easier than I expected.

I copied the backup'd BIOS to a USB drive to have it safe, in case i would need to flash it again. Then I took another copy of it to make modifications to it. This is done with the tool PolarisBiosEditor

Picture 2

You have to copy the value of the 1750 timings on the lower right of the PolarisBiosEditor to the 2000 timings and save it. Thats it. You have your own modified BIOS.

To flash the cards with the modified BIOS, I used ATIFLASH. I had to do it via console, because the program did not like 6 GPUs at once via GUI. This is the command to flash all cards with the given bios via cmd (run as administrator).

atiflash -pa biosname.rom

Now the cards have better/faster RAM timings. Its approximately a jump from 24 Mh/s to 28 Mh/s

This video helped me alot!

After flashing the BIOS i've made some tweaks to my GPUs via WattTool.

Picture 3

I've made changes to the yellow marked values in the picture above. First of all, its important to know, that Ethereum-Mining does not need a high GPU-Clock. It is much more RAM intensive than it is one the GPU. Thats why we can safe money by not wasteing it on a higher power bill. Reducing the Power Limit and the GPU clock itself reduces the power draw of the card.

Tuning the cards up to the sweetspot of maximum performance while being stable is a time-consuming task. All of my cards can run on -18 power limit, I would suggest you try reducing power draw first.

Set the state P4 on WattTool to 1100 and set the mV of P4 to P5-P7. For me, the value of P4's mV was around 1010-1070. Set the fanspeed to what ever you like, but keep in mind, a high rpm consumes more power than a lower one. This one depends on how hot it is where you live. I've set the Max Temp to 75°C, I'm assuming that the card will clock down once it reaches that temp, im not sure on this one. Reduce the Power Target as much as you can, -20 is the lowest you can do with this BIOS. Three out of my six cards can do -20. So the first step should be to go with -18 and see if it is stable.

If your rig is stable with those settings, you could go for an even lower Power Target. After that you should increase the Memory MHz. I increased it in +25MHz steps and once it got instable I reduced it by 5 MHz. By instable I mean a "Hang-Message" (see below).

These are my GPUs settings:

Miner-Console WattTool Physical Slot RAM MHz Power Limit Hashrate copied from console
GPU0 12 P4 2040 -19 28.546
GPU1 0 P5 2125 -18 29.059
GPU2 40 P1 2125 -20 29.290
GPU3 6 P6 2100 -20 28.952
GPU4 18 P3 2065 -19 28.582
GPU5 24 P2 2085 -20 28.718

Hang-Message:

...
19:01:28:280 1030 WATCHDOG: GPU 0 hangs in OpenCL call, exit
...

I've created a simple PowerShellScript. It reboots the computer once there is a "hang" message in the log file of the claymore miner.

$logfile = 'C:\Users\Miner01\Desktop\claymore\1494602312_log.txt'
Get-Content -Path $logfile -wait | %{if($_ -match "WATCHDOG: GPU"){shutdown /r /t 0}}
Unfortunately it does not determine which log file is the latest, so you would have to edit the file and change the $logfile-Variable to the latest log file. So the process would be like this. Start the miner via the start.bat, check the folder where the EthDcrMiner64.exe is located. Check which log is the latest (i.e. 1494602312_log.txt) , edit the PowerShell-Script und run it.

Now you would just see the Miner console and an empty blue'ish PowerShell-Console.

I've had trouble setting up the Task Scheduler to start the miner via "start.bat". Once thats running reliable, I'm going to share that aswell.




Picture 4




Picture 5

UPDATED:

  • in I: Using Claymore v9.1 ( -asm 0)
  • in II: created
  • in III: created

I hope you liked to read my story! I'm open for questions!
If you feel like you want to buy me a beer: 0xfD4d1B9ec7C75Af47787d3449d6e1f85c88B3eF1

Best wishes Dave2k


Comments

  • thepixelgeekthepixelgeek PhillyMember Posts: 96
    Thanks for the post! Great info.

    Re: hangs...Why not use a reboot.bat file with (shutdown /r /t 0), and copy start.bat file to shell:startup folder?

    Another option to WattTool would be to use Claymore config.txt file for all card settings.
  • BUTUZBUTUZ Member Posts: 139 ✭✭
    Why would there be hangs? If you are getting hangs then your card is unstable, voltage is too low.
  • Dave2kDave2k Member Posts: 46
    BUTUZ said:

    Why would there be hangs? If you are getting hangs then your card is unstable, voltage is too low.

    If im not mistaken, you cannot increase the voltage of the ram itself, you can only increase the voltage of the memory controller. So that would not necessarily make it more stable. (?)
  • Dave2kDave2k Member Posts: 46
    Dave2k said:

    BUTUZ said:

    Why would there be hangs? If you are getting hangs then your card is unstable, voltage is too low.

    If im not mistaken, you cannot increase the voltage of the ram itself, you can only increase the voltage of the memory controller. So that would not necessarily make it more stable. (?)

    ah, rethinking about this, you were talking about the voltage of the GPU itself. Your point is, too low power limit leads to hangs; while instable RAM Clocks lead to Incorrect shares, but not hangs. Is that it?
  • midoprincemidoprince Member Posts: 198 ✭✭
    good detailed post bro
    thanks
  • Dave2kDave2k Member Posts: 46


    Re: hangs...Why not use a reboot.bat file with (shutdown /r /t 0), and copy start.bat file to shell:startup folder?

    I tried the shell:startup-part, but it did start on logon, not on boot. There is a password on my account, otherwise you wouldn't be able to connect via RDP.
    I think i don't understand the first part tho. I want it to reboot automatically on a Hang-Message, because claymore does not always restart itself.
  • Dave2kDave2k Member Posts: 46

    Thanks for the post! Great info.

    Re: hangs...Why not use a reboot.bat file with (shutdown /r /t 0), and copy start.bat file to shell:startup folder?

    Another option to WattTool would be to use Claymore config.txt file for all card settings.

    AH i think i get it now. I didnt know there was a reboot option in claymore.
    This guy on reddit said the same like you, i assume.
    Sorry i didn't understand what you were trying to tell in the first place.
  • thepixelgeekthepixelgeek PhillyMember Posts: 96
    Cool!

    Fyi...I've tried adding "Load WattTool.exe name.ini name.ini name.ini" but it wasn't reliable. So, I opted for adding settings in the config.txt file in Claymore folder. Actually, I have all my settings in there.

    I will say, it's easier to use WattTool while tweaking cards to find stability...once you find it...add those setting in Claymore...

    Downside, is you can't step the settings for each card like WattTool.
  • Dave2kDave2k Member Posts: 46
    today, out of nowhere, windows updated my drivers and caused my miner to crash. Even though i had automatical driver installation disabled, as stated in the guide.

    I had to add this also: don't ask me why, Win10...
  • ksukhinksukhin Member Posts: 150 ✭✭
    @Dave2k I see that you're using SATA/Molex risers, I read your v1 post and you only powered 2 risers per cable coming from PSU.

    Have you had any issues with the cables overheating or burning? I have 20 of them and I'm getting a bunch of rx570s and I'm hesitant to use those because of so much info on them burning
  • Dave2kDave2k Member Posts: 46
    @ksukhin i could feel them getting warm. At first i had four cards/risers on one power line. You could definately feel the heat in the cable, so I decided to use only two, to be sure.
  • ksukhinksukhin Member Posts: 150 ✭✭
    Thank you for the quick reply, how long have you been running this rig for?
  • Dave2kDave2k Member Posts: 46
    ksukhin said:

    Thank you for the quick reply, how long have you been running this rig for?

    since early march.
  • DenguuDenguu Member Posts: 8
    Hey Man, thank you so much for the guide! It really helps! I'm having a bit of trouble getting my rig to boot up after enabling 4G, do you think you could help? It just stays on a black screen.
  • joel1145joel1145 Member Posts: 11
    What driver version are you running?
  • Dave2kDave2k Member Posts: 46
    @Denguu i've experienced that, when i had the wrong drivers, or more precise, when Windows 10 Updated the AMD drivers for me... but i could see the cards before the Update. Do you have a mix of 480 / 580? That might cause problems i saw somewhere on this forum.
    Dave2k said:

    today, out of nowhere, windows updated my drivers and caused my miner to crash. Even though i had automatical driver installation disabled, as stated in the guide.

    I had to add this also: don't ask me why, Win10...

  • DenguuDenguu Member Posts: 8
    My rig is made of 6 identical XFX RX 470s so I don't think that's the issue, I'm using the driver you said so 16.10.1 for AMD and 1.7 for the MB. I'm just really lost.
  • DenguuDenguu Member Posts: 8
    Maybe this will help?
    Rig Setup:
    Power: EVGA SuperNOVA 1200 P2
    CPU: Intel Pentium G4560
    RAM: Kingston HyperX FURY Black 8GB
    Storage: PNY CS1311 120GB SSD
    MB: Z170A Gaming Pro Carbon (Given lol)
    GPUS: 6x XFX Radeon RX 470 4GB
  • DenguuDenguu Member Posts: 8
    Also, do you just disable the updates for the videocards with the extra add on or is there something else?
  • DenguuDenguu Member Posts: 8
  • DenguuDenguu Member Posts: 8
    1.7 for the MB, 16.10.1 for Catalyst
  • Dave2kDave2k Member Posts: 46
    @Denguu i used this method, but thats apprently only for Win 10 Pro or similar.

    Did you increase the virtual memory of your machine? You have to set it to 16GB or higher.
  • DenguuDenguu Member Posts: 8
    That might be the issue..... Off to Amazon!
  • DenguuDenguu Member Posts: 8
    Sorry to ask so many times, but for the method you mentioned, just disable the video card updates correct?
  • Dave2kDave2k Member Posts: 46
    Denguu said:

    That might be the issue..... Off to Amazon!

    VIRTUAL Memory, the pagefile! Not Hardware RAM! Don't waste money :D
    Denguu said:

    Sorry to ask so many times, but for the method you mentioned, just disable the video card updates correct?

    There are 2 methods, either hide the windows update (that is for windows home) OR follow the guide i linked. You need to extract the hardware IDs and copy it into the policy. That is explained in the guide i linked here.

    If you dont have windows pro, check this post.

    If everything fails, use DDU and reinstall the drivers!
Sign In or Register to comment.