TraceView requires the installation of several components which collect and report the information you see on the dashboard. The first two are an instrumentation library called liboboe, and a daemon called Tracelyzer. Tracelyzer is the thing that connects to the TraceView service and reports back trace data. The methods in liboboe are used by our instrumentation modules to capture event and timing information as a request traverses your app.

The next component is web server instrumentation that captures apache httpd and nginx activity. This is not strictly necessary to take advantage of TraceView, but it will give you the most complete picture of application performance. You’ll also get real-user monitoring right out of the gate, instead of having to make additional code changes later.

Finally, there is the application language instrumentation, which is the code that makes tracing across all of our supported platforms and components possible. Nearly all of our webserver and language instrumentation is available as an add-on or module, with the exception of nginx which has no pluggable module support and so we offer a drop-in replacement. In any case all of the required TraceView code is already baked in, which means that you can get standard tracing up and running with just a few changes to your application environment.

Pre-requisites: The Tracelyzer needs to be able to connect with TraceView servers. Make sure you consult firewall and proxy settings, and be sure to consider whether you need a static setup, or are connecting via a SOCKS or web proxy.

Install via the dashboard

We’ve tried to make installing the requisite instrumentation components as simple as possible by enabling you to do this from the dashboard. Log in to your TraceView account login.tv.solarwinds.com and click on ‘install agents’. From there you’ll be led through a quick two-step installation process. You won’t be able to avoid the command line all together, but when possible, the exact command syntax is provided.

In cases where your environment is non-standard, the in-app install process might not work for you. For example if you compiled nginx from source or you installed php from dotdeb, use the self-guided instructions that follow. Installation is still quite simple, it just that it’ll be entirely command-line based. If you want to install instrumentation inside docker containers, those instructions are here.

traceview-install-agents.jpg

Self-guided installation

If you’ve determined that the TraceView installation via the dashboard isn’t going to work for you, this alternate self-guided method is basically the same two-step process plus a couple of checks to make sure everything went according to plan.

Define a new app

Initially TraceView won’t know how to organize incoming traces so the first thing you have to do is create a bucket for your traces. That bucket is called an ‘app’. When you define your app, TraceView automatically gives it an id. Later when you install base packages, you’ll specify that id go that all traces from the host.

  1. Navigate to > app configuration.
  2. Click ‘create new’.
  3. Name your app and save your changes.
  4. Note the token that was created for your app.

Install base packages

TraceView installation always starts out with installing a few base packages: the first is a daemon called Tracelyzer which connects to the TraceView service and reports trace data; the second is an instrumentation library called liboboe. The simplest way to install these packages is automated. For windows, download and run this installer. On linux just run our install script.

Update: The linux installer has been renamed to install_traceview.sh. While install_appneta.sh is still supported for backward compatibility, we encourage all customers to switch to the renamed installer.

wget [https://files.tv.solarwinds.com/install_traceview.sh | https://files.static.appneta.com/install_traceview.sh]
sudo sh ./install_traceview.sh [--static] <access-key> --app=<app_id>

Static options are provided if you intend on a static configuration.

Troubleshooting: The script provides user-friendly error messages if it doesn’t succeed. You can install these packages manually if you’re so inclined, or via a config management system if you have one.

Installed, running, and connected?

Before you move on to component instrumentation, it’s a good idea to check whether the Tracelyzer is connected to the TraceView service. The following steps apply to all supported operating systems, though the examples are for linux.

  1. The Tracelyzer should be running on each machine that should be reporting tracing data.

    sudo service tracelyzer status
    Checking status of tracelyzer...running.
    
  2. If it’s not listed, you’ll need to start it. Tracelyzer runs a series of diagnostics as part of initialization. In case of a failure it shows the reason on the console, and logs it to /var/log/tracelyzer/error.log. You can re-run the diagnostics anytime using ‘service traclyzer restart’.

    sudo /etc/init.d/tracelyzer start
    Starting tracelyzer...done.
    Testing connection (might take couple minutes):
      Testing SSH connection...success.
      Testing Tracelyzer TCP connection and UDP listening...success.
      Testing UDP connection to Tracelyzer...success.
    
  3. At this point we know that Tracelyzer is installed and running. Now we need to check that the TraceView service has registered it. For this we’ll use the one of the methods from our API.

    curl -G "https://api.tv.solarwinds.com/api-v2/hosts" -d key=<access-key>
    
  4. Finally, we’ll check that the app id you provided worked was effective in assigning your host to that app.

    curl -G "https://api.tv.solarwinds.com/api-v2/app/_app-name_/hosts" -d key=<access-key> | python -m json.tool
    

Component instrumentation

Installing instrumentation requires some familiarity with the components comprise your application. If you’re instrumenting php, python, or ruby, and don’t have apache or nginx in front of your app, contact support for additional guidance. You can skip this step for database hosts as they don’t need instrumentation.

Don’t forget to restart: Installation is not complete until you restart the instrumented component, e.g., jvm restart, apache restart, etc.

Webservers and load balancers

Application languages

Installed, running, and connected?

Before you start tracing, we need to check that the TraceView service registered all component instrumentation. In some cases you will have installed multiple packages, so rather than verifying each individually, let’s use the dashboard to check all of them at once.

  • From the TraceView dashboard select ‘Install Agents’.
  • Choose the components that correspond to what you installed, and move on to the next step.
  • Expand the drop-down in each step and verify that your host exists alongside the appropriate agents.

Troubleshooting: At this point, we’ve asked you to do all the basic checks, so if you’re not seeing all the components you expect, it’s time to give us a call.

Manual installation

Although our automated installer only recognizes officially supported distros, you can install on these other rpm-based distributions. If you do install on one of these system, please let us know! We’d love to know which versions work out-of-the-box, as well as any that might require modification.

Before you begin:

  1. You should only be using these instructions if the friendlier install methods won’t work because of some quirk in your application environment. What follows accomplishes the same thing as running the install script install_traceview.sh.
  2. Have your TraceView login credentials handy.
  3. Make sure to create the file /etc/tracelytics.conf and add to it this single line. This file enables unattended installation for Tracelyzer. Here are the instructions on how to find your access key.

    tracelyzer.access_key=<access-key>
    

rhel and centos

  1. Address the prerequisites.
  2. Add the appneta yum repository to your repo list by copying the matching snippet into a new file called /etc/yum.repos.d/appneta.repo. Remember to insert your access key.
  3. Download our rpm public gpg key file and place it in the directory /etc/pki/rpm-gpg.

    curl https://yum.tv.solarwinds.com/RPM-GPG-KEY-appneta > /etc/pki/rpm-gpg/RPM-GPG-KEY-appneta
    
  4. Check for package updates.

    sudo yum check-update
    
  5. Install the liboboe and Tracelyzer.

    sudo yum install liboboe liboboe-devel tracelyzer
    
  6. Run the configuration utility.

    sudo appneta-config
    
  7. Configure the system to run Tracelyzer at boot.

    sudo chkconfig --add tracelyzer
    
  8. Start Tracelyzer.

    sudo service tracelyzer start
    

64-bit centos/rhel 6

   [appneta]
   name=AppNeta
   baseurl=https://yum.tv.solarwinds.com/_access-key_/6/x86_64
   gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-appneta
   gpgcheck=1

64-bit centos/rhel 5

   [appneta]
   name=AppNeta
   baseurl=https://yum.tv.solarwinds.com/_access-key_/5/x86_64
   gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-appneta
   gpgcheck=1

64-bit Amazon Linux AMI

   [appneta]
   name=AppNeta
   baseurl=https://yum.tv.solarwinds.com/_access-key_/6/x86_64
   gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-appneta
   gpgcheck=1

32-bit centos/rhel 6

   [appneta]
   name=AppNeta
   baseurl=https://yum.tv.solarwinds.com/_access-key_/6/i386
   gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-appneta
   gpgcheck=1

32-bit centos/rhel 5

   [appneta]
   name=AppNeta
   baseurl=https://yum.tv.solarwinds.com/_access-key_/5/i386
   gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-appneta
   gpgcheck=1

32-bit Amazon Linux AMI

   [appneta]
   name=AppNeta
   baseurl=https://yum.tv.solarwinds.com/_access-key_/6/i386
   gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-appneta
   gpgcheck=1

Pinned centos/rhel 6 using openssl < 1.0.1e

If you’ve pinned your centos or rhel 6 system to pre-6.5 packages that don’t use openssl 1.0.1e, you’ll want to use our pinned component.

  1. Address the prerequisites.
  2. Add a new repo by creating the file /etc/yum.repos.d/appneta-6oldopenssl.repo with the following contents.

    [appneta-6oldopenssl]
    name=AppNeta 6oldopenssl
    baseurl=https://yum.tv.solarwinds.com/_access-key_/6oldopenssl/x86_64
    gpgcheck=1
    gpgkey=https://yum.tv.solarwinds.com/RPM-GPG-KEY-appneta
    enabled=1
    
  3. Update your yum metadata.

    yum makecache
    
  4. Determine whether the standard ‘appneta’ repository is already set up. If it is, either disable it in /etc/yum.repos.d/appneta.repo by setting ‘enabled=0’, or use the ‘–disablerepo=appneta’ option with yum install.

    yum list --showduplicates tracelyzer
    Available Packages
    ...
    tracelyzer.x86_64               1.1.6-1.el6                      appneta **&lt;-- disable this repo**
    tracelyzer.x86_64               1.1.6-1.el6                      appneta-6oldopenssl
    
  5. Install the instrumentation packages as you normally would, using the –disablerepo flag if you didn’t already disable the appneta repo in the previous step.

    yum install [--disablerepo=appneta tracelyzer]
    

debian and ubuntu

Our software is available for debian and ubuntu platforms from our apt repository. You’ll need to add our repository to your sources list and then install liboboe and Tracelyzer.

  1. Add our apt repository to your apt sources list by creating the file /etc/apt/sources.list.d/appneta.list with the following contents. Be sure to insert your access key, and your distribution codename, e.g., ‘precise’ or ‘wheezy’.

    sudo sh -c "echo deb https://apt.tv.solarwinds.com/_access-key distro_ main &gt; /etc/apt/sources.list.d/appneta.list"
    
  2. Tell apt to trust the gpg key we use to protect the integrity of our repository.

    curl https://apt.tv.solarwinds.com/appneta-apt-key.pub | sudo apt-key add -
    
  3. You can now update your local package lists and install our base packages.

    sudo apt-get update && sudo apt-get install liboboe0 liboboe-dev tracelyzer
    

dotdeb

We also support certain dotdeb releases. To enable access to these packages: add an extra dotdeb component at the end of the repository listing in /etc/apt/sources.list.d/appneta.list, as shown, and then run apt-get update.
deb https://apt.tv.solarwinds.com/access-key distro main dotdeb

Install with Chef or Puppet

If you are using a configuration management system like chef or puppet, have your recipes use our apt or yum repositories to install the base packages. You might also be interested in this chef cookbook for TraceView, which is our fork of a Tracelytics cookbook maintained by Sprintly. Our packages do not require user interaction if you have already created the file /etc/tracelytics.conf per the prerequisites.

TraceView + Docker

TraceView supports instrumenting applications that are inside Docker containers. Multiple containers can be run and instrumented simultaneously as long as network sharing is disabled. Its simply a matter of starting up the Tracelyzer service in each container.

If network sharing is enabled: If network sharing is enabled you might observe that applications are still instrumented and traces are generated in some the containers which share the network, but these results are spurious. Network sharing is disabled by default and is usually only enabled by using the –net flag with the ‘docker run’ command.

  1. Include the following instructions in your dockerfile:

    1. Obtain the script which will install the TraceView base packages.
      wget https://files.tv.solarwinds.com/install_traceview.sh
    2. Run install_traceview.sh script and pass it your TraceView account access key. Find your access key by logging in to TraceView, and navigating to your profile icon > ‘My Organization’.
      ./install_traceview.sh access-key
    3. Install instrumentation agents for your webserver and application language. See installation overview.
    4. Configure or customize instrumentation if desired.
    5. Start the Tracelyzer service.
      service tracelyzer start
    6. Start the application.
  2. Run the container and then verify that your instrumentation was properly installed.

Tracelyzer can’t bind to UDP socket…

The Tracelyzer runs a diagnostic check on startup, which uses netstat -p to verify Tracelyzer is listening on the required udp ports. By default Docker containers run without the privilege needed to allow netstat -p to work correctly, and so diagnostic check might fail citing ‘Tracelyzer can’t bind to UDP socket…’, even though the Tracelyzer is running. In this case, add the –cap-add SYS_PTRACE option.
   docker run [--cap-add 'SYS_PTRACE'] _mydockerimage_

1. From the TraceView dashboard select 'Install Agents'.
1. Choose the components that correspond to what you have installed, and move on to the next step.
1. Expand the drop-down in each step and verify that your container hostnames exist alongside the appropriate agents.
1. Generate some requests to your application and verify that the resulting traces are visible on your TraceView dashboard.
Example dockerfile
   FROM ubuntu:14.04
   RUN apt-get update && apt-get -y install wget curl git

   ##### INSTALL TRACELYZER AGENT #####
   RUN wget https://files.tv.solarwinds.com/install_traceview.sh
   RUN bash ./install_traceview.sh _access-key_
   CMD service tracelyzer start

   ##### INSTALL DJANGO AND INSTRUMENTATION #####
   RUN apt-get -y install python-pip python-dev build-essential
   ENV DJANGO_VERSION 1.8.2
   RUN pip install django==$DJANGO_VERSION

   RUN pip install oboe

   ##### DJANGO STACK #####
   ADD app /home/app/

   EXPOSE 8000