1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Tutorial: Install Logstash and Kibana on a Windows server.

Install Logstash and Kibana on a Windows server.

  1. sbagmeijer
    This guide should give you a good example of how to setup Logstash on a Windows environment for anyone that is not interested in using Linux.

    This guide was created by having all the applications on the same server, if you have different servers you have to think of the firewalls and you must alter the guide accordingly to fit your situation.

    This guide was tested on:
    Windows Server 2012 R2
    Windows 10 Professional (you have to adept certain steps yourself)

    In this guide I will show that it is also possible to run Logstash on a Windows machine and use IIS as web server.

    If your are looking for the Linux version of this guide please have a look here:
    How to install Logstash & Kibana on Fedora using Rsyslog as shipper.
    How to install Logstash & Kibana on Fedora using logstash-forwarder as shipper.

    So lets start!

    Download all requirements:
    [​IMG]
    Elasticsearch: https://download.elastic.co/elastic...p/elasticsearch/2.3.1/elasticsearch-2.3.1.zip
    Logstash: https://download.elastic.co/logstash/logstash/logstash-2.3.1.zip
    Kibana: https://download.elastic.co/kibana/kibana/kibana-4.5.0-windows.zip

    Shippers:
    Filebeat: https://download.elastic.co/beats/filebeat/filebeat-1.2.1-windows.zip
    Packetbeat: https://download.elastic.co/beats/packetbeat/packetbeat-1.2.1-windows.zip
    Topbeat: https://download.elastic.co/beats/topbeat/topbeat-1.2.1-windows.zip
    Winlogbeat: https://download.elastic.co/beats/winlogbeat/winlogbeat-1.2.1-windows.zip

    In short explained:
    Filebeat is for shipping log files to Logstash.
    Packetbeat is for analyzing your network data.
    Topbeat is for getting infrastructure information such as cpu and memory usage.
    Winlogbeat is for shipping windows event logs.

    Service manager:
    NSSM: https://nssm.cc/release/nssm-2.24.zip

    Beats Dashboard: http://download.elastic.co/beats/dashboards/beats-dashboards-1.2.1.zip.

    NSSM is required to make it so "Logstash" and "Kibana" can run as windows services.

    Feel free to download the newest versions, however be aware that Logstash is a continuous developed program and things might not work the same as I show below in the new version.

    You do not have to install all shippers you can simply use the one that will benefit you and skip the parts of the other shippers.

    Now just simply exact all the zip files to a folder that I created myself called "ulyaoth" in "c:\ulyaoth\" just o make things more easy.

    So for me it looks like this now:
    c:\ulyaoth\elasticsearch
    c:\ulyaoth\logstash
    c:\ulyaoth\kibana
    c:\ulyaoth\filebeat
    c:\ulyaoth\topbeat
    c:\ulyaoth\packetbeat
    c:\ulyaoth\winlogbeat
    c:\ulyaoth\nssm

    Please be aware this is the directory structure I use, if you change the naming of the directories or files you have to update everything correct in the guide to fit your situation.

    Prepare the server:
    Download the JDK version of Java and install it.
    Go to the java website: https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
    (Since Oracle updates Java frequently please download the latest and ignore if my version is slightly older in the images that follow)

    Accept the license and then download: "Windows x64 (jdk-8u77-windows-x64.exe)" package.

    Now install it by double clicking the file.
    [​IMG]
    If you see this warning press "Run" if you are sure to have downloaded the right file.

    You should now see:
    [​IMG]
    On this window press "Next" to see the get to the next window.

    [​IMG]
    On this windows press again on the "Next" button to come to the next window.

    [​IMG]
    Here you have you can select the destination folder where Java will be installed for me the default location is fine if for you also press "Next" to go to the next window or press "Change" to update the location and then press "Next".

    [​IMG]
    This window means it is installing Java so wait till it completes and you will see a new window.

    [​IMG]
    Java is now installed so press the "Close" button.

    Now let's add the JAVA_HOME variable to the server by simply right clicking on "This PC" and choose "Properties" then follow the below steps.
    [​IMG]
    So if you see click the picture to enlarge it you see a small 1,2,3,4,5 in red you simply follow that order.

    1. Click on "Change Settings.
    2. Click on "Advanced".
    3. Click on "Environment Variables.
    4. Click on "New".

    Now fill in the new window that appears like I did:
    Variable Name: JAVA_HOME
    Variable value: C:\Program Files\Java\jdk1.8.0_77

    5. Click on "OK" (You might need to Click on "OK" on the other windows also)

    That is all you should have to do.

    Prepare IIS:
    Windows Server 2012 R2:
    Open Server Manager:
    [​IMG]
    Click on "2 Add roles and features" the on I placed a red box around on the image above and a new window should appear:

    [​IMG]
    On this page press on "Next" to go to the next window.

    [​IMG]
    On this page leave the default selection "Role-based or feature-based installation and press "Next" again.

    [​IMG]
    For me all settings are fine here if that is the case for you also then press on "Next" again to go to the next window.

    [​IMG]
    On this window scroll down till you see "Web Server (IIS) the one I put a small red box around and then click the select box and then a new window should popup like this:

    [​IMG]
    Simply press the "Add Features" button and you come back at the previous image where you can now press "Next".

    [​IMG]
    Here you can choose to install additional features however we do not require it for this guide so simply press "Next", and on the next informative page press "Next" again.

    [​IMG]
    On this page you can select options for your IIS webserver you can go trough them and select what you like, normally I at-least select "HTTP Redirection" as it is a useful option, you can select it by clicking the selection box.

    if you need nothing else press "Next" again.

    [​IMG]
    This confirmation window shows everything that will be installed so double check it so you do not install things you do not wish and then press the "Install" button to start the installation and you will see the following window:

    [​IMG]
    Just let it go and when the bar is fully blue and says it is finished press the "Close" button.

    IIS is now fully installed and ready to be used.

    Windows 10:
    On Windows 10 you should go to your "Control Panel" then go to "Program and Features" and then click on "Turn Windows features on or off" here you will find "Internet Information Service" which is IIS.

    Now we also have to install Microsoft Web Platform Installer 5.0 so go to the following website and download it: http://www.microsoft.com/web/downloads/platform.aspx

    Once you have downloaded the file called "wpilauncher.exe" simply click it and after a while the app opens and in the right top corner you have a search box.

    In the search box type ARR and press the enter key, you will see a list of applications that matches your search. The one we are interested in is "Application Request Routing 3.0" so find it in the list and press the "Add" button behind it:
    [​IMG]

    Then use the search box again in the top but this time search for "URL Rewrite" you will see then the application "URL Rewrite 2.o" now press again the "Add" button behind it:
    [​IMG]

    Now you have added the two additional applications we need you can go ahead and press the "Install" button and the following popup will appear:

    [​IMG]
    If everything looks fine as you selected press the "I Accept" button to start the installation it should look like this by now:

    [​IMG]
    Once this window is finished it should show a new window confirming that everything is installed:

    [​IMG]
    Just press on "Finish" to complete the installation and then on "Exit" to close the whole app.

    Now open IIS Manager and you should see a window as below.
    [​IMG]
    Version could be different depending on the Windows version you are using.

    From this window we will create a new website for Kibana in IIS.
    Right click on "sites" in the left part of IIS Manager and click "Add Website" as you see on the image below:
    [​IMG]
    Once you click it you should get a popup window like the image below.

    [​IMG]
    You should fill it in as I did above with of-course "your" information, if you wish to have SSL then change the "Type" field to "https" and select your SSL certificate at the option that will show.

    it is important to know the site and host name this is something you have to alter to your own naming, if you want to use my name you have to update you hosts file:

    Open a powershell and type the following commands:
    1. cd C:\Windows\System32\drivers\etc
    2. notepad hosts

    It should open notepad and in the bottom just add this:
    127.0.0.1 loghost.ulyaoth.net

    It should look like this:
    [​IMG]

    And as final part we have to setup a reverse proxy in IIS to Kibana.
    Click on your newly created name in the left window mine is called "loghost.ulyaoth.net" and then in the left window allot of icons will appear:
    [​IMG]

    Here double click on the "Rewrite URL" button the on I added a red box around and then you should see the following window:
    [​IMG]

    On this window click in the right menu on "Add Rules(s)..." the one I again added a red box around and you should see the following popup window:

    [​IMG]

    On this screen click on "Reverse Proxy" and then in the bottom on the "OK" button if this is your first installation or try you will get a warning like this:

    [​IMG]
    Read the warning to understand the impact of doing a reverse proxy and then press the "OK" button since we require this functionality and again a new window appears.

    [​IMG]
    On this windows make sure you fill everything in exactly as I did above:
    Inbound Rules: 127.0.01:5601

    If you did this you should have everything as I did so go ahead and press on the "OK" button.

    To explain it in short the ip you entered "127.0.0.1" and port "5601" this is where Kibana will run on your server, since in my personal opinion it is not recommended to run Kibana directly towards the internet for security reasons and or you might need additional futures you let IIS be the link in-between since this is a well tested web-server.

    You have come to the point where you have prepared the server by installing Java, IIS and created the reverse proxy for Kibana.

    In the next part we will configure and start the ELK stack.

    Configure and start ELK stack:
    1. Elasticsearch:

    Open powershell and type the following commands:
    Code:
    cd c:\ulyaoth\elasticsearch\bin\
    .\service install
    
    It should look like this in your powershell.
    [​IMG]

    Now continue by typing the following:
    Code:
    .\service manager
    
    You should see the Elasticsearch service manager:
    [​IMG]

    You have to change on the tab the "Startup type" from Manual to Automatic and then press Apply this should make Elasticsearch start automatically on server boot.

    This window contains some more options such as how much memory Elasticsearch will use you can find this on the "Java" tab I would suggest to make this fitting for your server if you have a servers that will handle a huge amount of logs I would increase the "Maximum Memory Pool: 1024" at least to a higher amount.

    Before you close the window make sure to press "Start" so it actually will run right now :)

    This is everything to start ElasticSearch automatically on boot to test that it is working open a browser and go to the url "127.0.0.1:9200".

    If you see a json string something like what you see below in the picture then it means your ElasticSearch is running:
    [​IMG]
    (I user Firefox Developers edition that is why it looks black, yours probably shows as a white site).

    2. Logstash:
    Download the required configuration file for Logstash from our github.

    Do this by opening powershell and then run the following command:
    Code:
    Start-BitsTransfer -Source https://raw.githubusercontent.com/ulyaoth/tutorials/master/configs/windows/logstash/logstash.json -Destination C:\ulyaoth\logstash\bin\
    
    Your powershell should look like this:
    [​IMG]
    This should place the file called "logstash,json" in the directory "C:\ulyaoth\logstash\bin\".

    make sure to open the actual config and see if you need to change anything that will fit your situation or naming.

    We will now use the downloaded NSSM from the start of the guide to create a service for Logstash.

    Now open powershell and type the following command:
    Code:
    C:\ulyaoth\nssm\win64\nssm install Logstash
    
    Like this:
    [​IMG]

    You will now see a GUI to create a service on the first tab "Application" fill in the following:
    Path: C:\ulyaoth\logstash\bin\logstash.bat
    Startup directory: C:\ulyaoth\logstash\bin
    Arguments: -f C:\ulyaoth\logstash\bin\logstash.json

    It should look like this:
    [​IMG]

    If everything looks the same then go to the "Details" tab make sure it looks the same as below:
    [​IMG]

    Now the following step is additional but recommended if you are 100% going to use Elasticsearch, click on the "Dependencies" tab and add the following: elasticsearch-service-x64
    [​IMG]
    The reason for this step is that Logstash can do difficult if you have a Elasticsearch output but it is not actually running.

    Now this is all so press on the "Install service" button to finish setting up a service for Logstash.
    [​IMG]

    If you wish to adjust java related Logstash settings such as memory you can do this in the file "logstash\bin\setup.bat".

    As final step we require the beats plugin so we can run our shippers.
    Open a "Command Prompt" or a "powershell" window and run the following command:
    Code:
    C:\ulyaoth\logstash\bin\plugin install logstash-input-beats
    
    You should see this:
    [​IMG]

    3. Kibana
    We will once again use NSSM to create a service for Kibana so start with opening powershell and running the following command:
    Code:
    C:\ulyaoth\nssm\win64\nssm install Kibana
    
    Like this:
    [​IMG]

    You will now see a GUI to create a service on the first tab "Application" fill in the following:
    Path: C:\ulyaoth\kibana\bin\kibana.bat
    Startup directory: C:\ulyaoth\kibana\bin

    It should look like this:
    [​IMG]

    If everything looks the same then go to the "Details" tab make sure it looks the same as below:
    [​IMG]

    Now the following step is additional but recommended click on the "Dependencies" tab and add dependencies for:
    elasticsearch-service-x64
    Logstash
    [​IMG]
    The reason for this step is that Logstash can do difficult if you have a Elasticsearch output but it is not actually running.

    Now this is all so press on the "Install service" button to finish setting up a service for Kibana.
    [​IMG]

    If you wish to adjust the settings of Kibana such as running it on a different port or IP simply go to "C:\ulyaoth\kibana\config\kibana.yml" and play around with the available settings.

    This was all to have ELK installed now let's make sure all services are actually started.

    Simply open "Services" and make sure the following services are started:
    Elasticsearch
    Logstash
    Kibana

    Shippers:
    All the Beats programs are installed the same way so I can simply show it once and then you know how to install them.

    Open Powershell and run the following commands to make services from all the beats programs:
    Code:
    PowerShell.exe -ExecutionPolicy UnRestricted -File C:\ulyaoth\filebeat\.\install-service-filebeat.ps1
    
    PowerShell.exe -ExecutionPolicy UnRestricted -File C:\ulyaoth\topbeat\.\install-service-topbeat.ps1
    
    PowerShell.exe -ExecutionPolicy UnRestricted -File C:\ulyaoth\packetbeat\.\install-service-packetbeat.ps1
    
    PowerShell.exe -ExecutionPolicy UnRestricted -File C:\ulyaoth\winlogbeat\.\install-service-winlogbeat.ps1
    
    At the "Security warning" choose "R" for run once.

    It should look like this:
    [​IMG]

    Now let's download a newer configuration file for filebeat so it will ship the IIS logs to Kibana found on our github.

    Open PowerShell and run the following command:
    Code:
    Start-BitsTransfer -Source https://raw.githubusercontent.com/sbagmeijer/tutorials/master/configs/windows/logstash/filebeat.yml -Destination C:\ulyaoth\filebeat\
    
    Start-BitsTransfer -Source https://raw.githubusercontent.com/sbagmeijer/tutorials/master/configs/windows/logstash/topbeat.yml -Destination C:\ulyaoth\topbeat\
    
    Start-BitsTransfer -Source https://raw.githubusercontent.com/sbagmeijer/tutorials/master/configs/windows/logstash/packetbeat.yml -Destination C:\ulyaoth\packetbeat\
    
    Start-BitsTransfer -Source https://raw.githubusercontent.com/sbagmeijer/tutorials/master/configs/windows/logstash/winlogbeat.yml -Destination C:\ulyaoth\winlogbeat\
    
    It should look like this in your powershell:
    [​IMG]

    In your beats folders you will find the files makes sure to read trough them so you understand it and so you can add additional things:
    C:\ulyaoth\filebeat\filebeat.yml
    C:\ulyaoth\topbeat\topbeat.yml
    C:\ulyaoth\packetbeat\packetbeat.yml
    C:\ulyaoth\winlogbeat\winlogbeat.yml

    Make sure to open all configuration files ant to tweak them to your needs, all shippers are installed with default values you have to configure it further for own specifics.

    Additional step for Packetbeat:
    In order to run Packetbeat you also must install the program called "WinPcap" so go ahead and download this here:
    WinPcap: http://www.winpcap.org/install/bin/WinPcap_4_1_3.exe

    Double click the executable "WinPcap_4_1_3.exe" and you should see the following window:

    [​IMG]
    Press on the button "Next" to continue and you will see the following window:

    [​IMG]
    Make sure to read the full agreement and if you agree then press on the button "I Agree" and a new window will appear:

    [​IMG]
    On this screen you can choose if you wish the service to autostart on boot or not, by default this is enabled and it is recommended you leave this since else Packetbeat would not start automatically, so continue and press the "Install" button.

    The installation should be very quick and you will probably instantly see the following window:

    [​IMG]
    This means it installed successfully so press the "Finish" button to close down the program.

    Now that you have installed all Beat services you have to make sure they are all started open the program "services.msc" and now start the programs:
    • filebeat
    • topbeat
    • packetbeat
    • winlogbeat
    Setup Kibana:
    Congratulations you are now at the final part and all that is left is to configure Kibana so you will see the results of your hard work!

    If you did everything correct then Kibana should now be running so lets test it by going to "http://loghost.ulyaoth.net/" or the website name you did choose and you should see that Kibana is started:
    [​IMG]

    Now by default this will not work since the indexes we use are called after the beats so change the "Index name or pattern" to "filebeat-*" and no worries the other beat we configure later.
    [​IMG]
    So when you have filled everything in exactly as me above then press on "Create".

    You now should see a window like this:
    [​IMG]

    Now since we only created the "filebeat-*" index we have to make sure that Kibana does know of the other ones so on the image above press on the link left top corner called "+Add New" I did place a red box around it on the image above.

    You should now see the window:
    [​IMG]
    You should see "logstash-*" so change it to "topbeat-*" like I did above.

    Now repeat this process and add also:
    packetbeat-*
    winlogbeat-*

    If you did it all correct your right top corner should now look like this:
    [​IMG]

    Well that is all to configure Kibana now click in the top on "Discover".

    By default your "filebeat-*" index will only show so you should see your Kibana IIS logs:
    [​IMG]

    You can change trough your different indexes by clicking in your right top corner on the black bar that contains "filebeat-*" and then change your index to one of the other beats.

    CONGRATULATIONS! You have a ELK stack running on Windows Server.

    Extras!:

    Elasticsearch is kind enough to provide predefined dashboards that are very simple to install.

    Download the following file:
    beats-dashboards: http://download.elastic.co/beats/dashboards/beats-dashboards-1.2.1.zip

    Unzip the folder and then run with powershell the following commands:
    cd C:\Users\Administrator\Downloads\beats-dashboards-1.2.1\

    PowerShell.exe -ExecutionPolicy UnRestricted -File .\load.ps1

    Make sure to "cd" to your extraction location.

    When that finished running it should look like this in powershell:
    [​IMG]

    Now open kibana then follow the picture below:
    [​IMG]
    1. Click on "Dashboard"
    2. Click on "Load saved Dashboard"
    3. Type "Topbeat-Dashboard" (example)

    Now click on the result and it should open a dashboard for you predefined:
    [​IMG]

    If it is a new install you maybe not have enough data so play around with the time settings and just in general they provided many other predefined dashboards.

    have fun!.

    Additional Troubleshooting Tips:
    1. Always check in CMD if things not work, Logstash, Kibana and Elasticsearch give output in a terminal if you run it locally and this helps allot!.
    2. Always check in "services.msc" is everything actually running I get many questions and then ES is not running for example.
    3. Make sure your filebeat points to the correct log files.
    4. Keep the versions of ELK intact as I write, many times there are updates that only work with certain versions of Elasticsearch for example.

    If you have any suggestions to improve this guide then please feel free to or update the configs on GitHub or to provider me the information so I can update the guide so it can help others!, just click the "Ask Questions / Get Support" button.

    [​IMG]
    Naman and Gohar like this.

Recent Reviews

  1. bellecci
    bellecci
    5/5,
    You will have ELK up and running successfully in your environment at the end of this tutorial. Excellent
  2. Naman
    Naman
    4/5,
    I am new to ELKB. Please tell me why Elasticsearch was not configured using nssm
    1. sbagmeijer
      Author's Response
      Because Elasticsearch provides a own service solution, so there is no need to use a 3th party solution like nssm.
  3. Gohar
    Gohar
    4/5,
    Can you please help me change IP from 127.0.0.1 to myserverIP as I cannot send logs from other servers? Thanks.
    1. sbagmeijer
      Author's Response
      What IP are you revering to?

      Logstash should automatically run on your network cards ip, so all you should have to do is open your firewalls to allow other corporate ip addresses to access that specific port.

      Then you can install one of the beat or nxlogs on any other pc and simply ship to the servers ip/por of your logstash server.