Merge pull request #631 from nautobot/u/nopg/solarwinds-integration

Add SolarWinds SSoT Integration

.github/CODEOWNERS

@@ -12,3 +12,5 @@
 /nautobot_ssot/integrations/meraki/ @jdrew82 @nautobot/plugin-ssot
 /nautobot_ssot/integrations/servicenow/ @glennmatthews @qduk @nautobot/plugin-ssot
 /nautobot_ssot/integrations/slurpit/ @lpconsulting321 @pietos @nautobot/plugin-ssot
+/nautobot_ssot/integrations/solarwinds/ @jdrew82 @nopg @nautobot/plugin-ssot
+

README.md

@@ -38,6 +38,7 @@ This Nautobot application framework includes the following integrations:
 - Cisco Meraki
 - ServiceNow
 - Slurpit
+- SolarWinds
 
 Read more about integrations [here](https://docs.nautobot.com/projects/ssot/en/latest/user/integrations). To enable and configure integrations follow the instructions from [the install guide](https://docs.nautobot.com/projects/ssot/en/latest/admin/install/#integrations-configuration).
 
@@ -95,6 +96,7 @@ The SSoT framework includes a number of integrations with external Systems of Re
 * Cisco Meraki
 * ServiceNow
 * Slurpit
+* SolarWinds
 
 > Note that the Arista CloudVision integration is currently incompatible with the [Arista Labs](https://labs.arista.com/) environment due to a TLS issue. It has been confirmed to work in on-prem environments previously.
 

changes/631.added

@@ -0,0 +1 @@
+Added integration with SolarWinds.

changes/631.changed

@@ -0,0 +1 @@
+Added documentation for SolarWinds integration.

development/creds.example.env

@@ -36,6 +36,9 @@ NAUTOBOT_SSOT_CITRIX_ADM_PASSWORD="changeme"
 
 NAUTOBOT_SSOT_INFOBLOX_PASSWORD="changeme"
 
+NAUTOBOT_SSOT_SOLARWINDS_USERNAME="admin"
+NAUTOBOT_SSOT_SOLARWINDS_PASSWORD="changeme"
+
 # ACI Credentials.  Append friendly name to the end to identify each APIC. 
 NAUTOBOT_APIC_BASE_URI_NTC=https://aci.cloud.networktocode.com
 NAUTOBOT_APIC_USERNAME_NTC=admin

development/development.env

@@ -115,4 +115,6 @@ IPFABRIC_TIMEOUT=15
 NAUTOBOT_SSOT_ENABLE_ITENTIAL="True"
 
 NAUTOBOT_SSOT_ENABLE_SLURPIT="False"
-SLURPIT_HOST="https://sandbox.slurpit.io"
+SLURPIT_HOST="https://sandbox.slurpit.io"
+
+NAUTOBOT_SSOT_ENABLE_SOLARWINDS="False"

development/nautobot_config.py

@@ -232,6 +232,7 @@
         "enable_meraki": is_truthy(os.getenv("NAUTOBOT_SSOT_ENABLE_MERAKI")),
         "enable_servicenow": is_truthy(os.getenv("NAUTOBOT_SSOT_ENABLE_SERVICENOW")),
         "enable_slurpit": is_truthy(os.getenv("NAUTOBOT_SSOT_ENABLE_SLURPIT")),
+        "enable_solarwinds": is_truthy(os.getenv("NAUTOBOT_SSOT_ENABLE_SOLARWINDS")),
         "hide_example_jobs": is_truthy(os.getenv("NAUTOBOT_SSOT_HIDE_EXAMPLE_JOBS")),
         "device42_defaults": {
             "site_status": "Active",

docs/admin/install.md

@@ -100,3 +100,4 @@ Set up each integration using the specific guides:
 - [Cisco Meraki](./integrations/meraki_setup.md)
 - [ServiceNow](./integrations/servicenow_setup.md)
 - [Slurpit](./integrations/slurpit_setup.md)
+- [SolarWinds](./integrations/solarwinds_setup.md)

docs/admin/integrations/index.md

@@ -14,3 +14,4 @@ This Nautobot app supports the following integrations:
 - [Cisco Meraki](./meraki_setup.md)
 - [ServiceNow](./servicenow_setup.md)
 - [Slurpit](./slurpit_setup.md)
+- [SolarWinds](./solarwinds_setup.md)

docs/admin/integrations/solarwinds_setup.md

@@ -0,0 +1,25 @@
+# SolarWinds Integration Setup
+
+This guide will walk you through steps to set up the SolarWinds integration with the `nautobot_ssot` app.
+
+## Prerequisites
+
+Before configuring the integration, please ensure, that `nautobot-ssot` app was [installed with the SolarWinds integration extra dependencies](../install.md#install-guide).
+
+```shell
+pip install nautobot-ssot[solarwinds]
+```
+
+## Configuration
+
+Access to your SolarWinds instance is defined using the [ExternalIntegration](https://docs.nautobot.com/projects/core/en/stable/user-guide/platform-functionality/externalintegration/) model which allows you to utilize this integration with multiple instances concurrently. Please bear in mind that it will synchronize all data 1:1 with the specified instance to match exactly, meaning it will delete data missing from an instance. Each ExternalIntegration must specify a SecretsGroup with [Secrets](https://docs.nautobot.com/projects/core/en/stable/user-guide/platform-functionality/secret/) that contain the SolarWinds administrator Username and Password to authenticate with. You can find Secrets and SecretsGroups available under the Secrets menu.
+
+Below is an example snippet from `nautobot_config.py` that demonstrates how to enable the SolarWinds integration:
+
+```python
+PLUGINS_CONFIG = {
+    "nautobot_ssot": {
+        "enable_solarwinds": is_truthy(os.getenv("NAUTOBOT_SSOT_ENABLE_SOLARWINDS", "true")),
+    }
+}
+```

docs/user/integrations/index.md

@@ -14,3 +14,4 @@ This Nautobot app supports the following integrations:
 - [Cisco Meraki](./meraki.md)
 - [ServiceNow](./servicenow.md)
 - [Slurpit](./slurpit.md)
+- [SolarWinds](./solarwinds.md)

docs/user/integrations/solarwinds.md

@@ -0,0 +1,110 @@
+# SolarWinds SSoT Integration
+
+The SolarWinds integration is built as part of the [Nautobot Single Source of Truth (SSoT)](https://github.com/nautobot/nautobot-app-ssot) app. The SSoT app enables Nautobot to be the aggregation point for data coming from multiple systems of record (SoR).
+
+From SolarWinds into Nautobot, it synchronizes the following objects:
+
+| SolarWinds              | Nautobot                     |
+| ----------------------- | ---------------------------- |
+| Container               | Location*                    |
+| Devices                 | Devices                      |
+| Vendor                  | Manufacturers                |
+| Model/DeviceType        | DeviceTypes                  |
+| Model/Vendor            | Platforms                    |
+| Versions                | SoftwareVersions             |
+| Interfaces              | Interfaces                   |
+| IP Addresses            | IP Addresses                 |
+
+## Usage
+
+Once the app is installed and configured, you will be able to perform an inventory ingestion from SolarWinds Orion into Nautobot. From the Nautobot SSoT Dashboard view (`/plugins/ssot/`), or via Apps -> Single Source of Truth -> Dashboard, SolarWinds will show as a Data Source.
+
+![Dashboard View](../../images/solarwinds_dashboard.png)
+
+From the Dashboard, you can also view more information about the App by clicking on the `SolarWinds to Nautobot` link and see the Detail view. This view will show the mappings of SolarWinds objects to Nautobot objects, the sync history, and other configuration details for the App:
+
+![Detail View](../../images/solarwinds_detail-view.png)
+
+In order to utilize this integration you must first enable the Job. You can find the available installed Jobs under Jobs -> Jobs:
+
+![Job List](../../images/solarwinds_job_list.png)
+
+To enable the Job you must click on the orange pencil icon to the right of the `SolarWinds to Nautobot` Job. You will be presented with the settings for the Job as shown below:
+
+![Job Settings](../../images/solarwinds_job_settings.png)
+
+You'll need to check the `Enabled` checkbox and then the `Update` button at the bottom of the page. You will then see that the play button next to the Job changes to blue and becomes functional, linking to the Job form.
+
+![Enabled Job](../../images/solarwinds_enabled_job.png)
+
+Once the Job is enabled, you'll need to manually create a few objects in Nautobot to use with the Job. First, you'll need to create a Secret that contains your SolarWinds username and Password for authenticating to your desired SolarWinds instance:
+
+![Username Secret](../../images/solarwinds_username_secret.png)
+
+![Password Secret](../../images/solarwinds_password_secret.png)
+
+Once the required Secrets are created, you'll need to create a SecretsGroup that pairs them together and defines the Access Type of HTTP(S) like shown below:
+
+![SolarWinds SecretsGroup](../../images/solarwinds_secretsgroup.png)
+
+With the SecretsGroup defined containing your instance credentials you'll then need to create an ExternalIntegration object to store the information about the SolarWinds instance you wish to synchronize with.
+
+![SolarWinds ExternalIntegration](../../images/solarwinds_external_integration.png)
+
+> The only required portions are the Name, Remote URL, Verify SSL, HTTP Method (GET), and Secrets Group. 
+- The External Integration will need it's `http_method` set to `GET`. 
+- Keep the `verify_ssl` setting in mind, uncheck this if you are using untrusted certificates
+
+Extra settings can be configured in the Extra Config section of your External Integration, example below:
+
+| Setting         | Default | Description                                                                       |
+| --------------- | ------- | --------------------------------------------------------------------------------- |
+| port            |   17774 | TCP port used for communication to the API                                        |
+| retries         |       5 | How many retries before considering the connection to Solarwinds failed           |
+| batch_size      |     100 | How many nodes to include in queries, this can be lowered to prevent API timeouts |
+
+```json
+{
+    "port": 443,
+    "retries": 10,
+    "batch_size": 100
+}
+```
+
+With those configured, you will then need to ensure you have the Locations and Location Types defined to be used for the imported Devices. With those created, you can run the Job to start the synchronization:
+
+![Job Form](../../images/solarwinds_job_form.png)
+
+If you wish to just test the synchronization but not have any data created in Nautobot you'll want to select the `Dryrun` toggle. Clicking the `Debug` toggle will enable more verbose logging to inform you of what is occuring behind the scenes. After those toggles there are also dropdowns that allow you to specify the SolarWinds instance to synchronize with and to define the LocationType to use for the imported Devices from SolarWinds. In addition, there are also some optional settings on the Job form:
+
+- You can choose to pull all devices from a specific SolarWinds Container (and subcontainers), or you can use a SolarWinds CustomProperty. This CustomProperty should be a Boolean set to `True`, and assigned to all devices you wish to sync. Enter the name of this CustomProperty into the CustomProperty field.
+- If pulling from CustomProperty, you must choose the Location to place devices using the Location Override option, and should still choose the Container or ALL Containers.
+- If the LocationType that you specify for the imported Devices requires a parent LocationType to be assigned, you must also select the Parent LocationType.
+
+
+In addition, there are a few methods provided to assign Roles to your imported Devices. You can choose a Default Role to be used for all Devices not mapped via a method below.
+
+The Role Matching Attribute can be set to DeviceType or Hostname. You then provide a `role_map` to associate specific DeviceTypes or Hostnames to a Role name. This map should be a standard python dictionary if using DeviceType. Regex can be used to match Hostnames. Examples below:
+
+```python title="Role_Map using DeviceType"
+{
+    "C8300": "ROUTER_ROLE",
+    "C9200": "SWITCH_ROLE"
+}
+```
+```python title="Role_Map using Hostname and Regex"
+{
+    "CORE.*": "ROUTER_ROLE", 
+    "WLC.*": "WLC_ROLE"
+}
+```
+
+- Finally there is an option to specify a Tenant to be assigned to the imported Devices, Prefixes, and IPAddresses. This is handy for cases where you have multiple SolarWinds instances that are used by differing business units.
+!!! info
+    Tenant names will also be used as Namespace names for any IP Addresses and Prefixes created, these Namespaces must be created by you!
+
+Running this Job will redirect you to a `Nautobot Job Result` view.
+
+![JobResult View](../../images/solarwinds_jobresult.png)
+
+Once the Job has finished you can click on the `SSoT Sync Details` button at the top right of the Job Result page to see detailed information about the data that was synchronized from SolarWinds and the outcome of the sync Job. There are also more logs to check if you run into issues, that can be found under Apps -> Logs.

mkdocs.yml

@@ -120,6 +120,7 @@ nav:
           - Cisco Meraki: "user/integrations/meraki.md"
           - ServiceNow: "user/integrations/servicenow.md"
           - Slurpit: "user/integrations/slurpit.md"
+          - SolarWinds: "user/integrations/solarwinds.md"
       - Modeling: "user/modeling.md"
       - Performance: "user/performance.md"
       - Frequently Asked Questions: "user/faq.md"
@@ -139,6 +140,7 @@ nav:
           - Itential: "admin/integrations/itential_setup.md"
           - Cisco Meraki: "admin/integrations/meraki_setup.md"
           - ServiceNow: "admin/integrations/servicenow_setup.md"
+          - SolarWinds: "admin/integrations/solarwinds_setup.md"
           - Slurpit: "admin/integrations/slurpit_setup.md"
       - Upgrade: "admin/upgrade.md"
       - Uninstall: "admin/uninstall.md"

nautobot_ssot/__init__.py

@@ -26,6 +26,7 @@
     "nautobot_ssot_itential",
     "nautobot_ssot_meraki",
     "nautobot_ssot_servicenow",
+    "nautobot_ssot_solarwinds",
 ]
 
 
@@ -107,8 +108,10 @@ class NautobotSSOTAppConfig(NautobotAppConfig):
         "enable_citrix_adm": False,
         "enable_infoblox": False,
         "enable_ipfabric": False,
+        "enable_meraki": False,
         "enable_servicenow": False,
         "enable_slurpit": False,
+        "enable_solarwinds": False,
         "enable_itential": False,
         "hide_example_jobs": True,
         "ipfabric_api_token": "",

nautobot_ssot/integrations/solarwinds/__init__.py

@@ -0,0 +1 @@
+"""Base module for Solarwinds integration."""

nautobot_ssot/integrations/solarwinds/constants.py

@@ -0,0 +1,24 @@
+"""Constants to be used with Solarwinds SSoT."""
+
+ETH_INTERFACE_NAME_MAP = {
+    "AppGigabitEthernet": "virtual",
+    "FastEthernet": "100base-tx",
+    "GigabitEthernet": "1000base-t",
+    "FiveGigabitEthernet": "5gbase-t",
+    "TenGigabitEthernet": "10gbase-t",
+    "TwentyFiveGigE": "25gbase-x-sfp28",
+    "FortyGigabitEthernet": "40gbase-x-qsfpp",
+    "FiftyGigabitEthernet": "50gbase-x-sfp28",
+    "HundredGigE": "100gbase-x-qsfp28",
+}
+
+ETH_INTERFACE_SPEED_MAP = {
+    "100Mbps": "100base-tx",
+    "1Gbps": "1000base-t",
+    "5Gbps": "5gbase-t",
+    "10Gbps": "10gbase-t",
+    "25Gbps": "25gbase-x-sfp28",
+    "40Gbps": "40gbase-x-qsfpp",
+    "50Gbps": "50gbase-x-sfp28",
+    "100Gbps": "100gbase-x-qsfp28",
+}

nautobot_ssot/integrations/solarwinds/diffsync/__init__.py

@@ -0,0 +1 @@
+"""DiffSync adapters and models for Solarwinds SSoT."""

nautobot_ssot/integrations/solarwinds/diffsync/adapters/__init__.py

@@ -0,0 +1 @@
+"""Adapter classes for loading DiffSyncModels with data from Solarwinds or Nautobot."""

nautobot_ssot/integrations/solarwinds/diffsync/adapters/nautobot.py

@@ -0,0 +1,54 @@
+# pylint: disable=duplicate-code
+"""Nautobot Adapter for Solarwinds SSoT app."""
+
+from nautobot_ssot.contrib.adapter import NautobotAdapter as BaseNautobotAdapter
+from nautobot_ssot.integrations.solarwinds.diffsync.models.base import (
+    DeviceModel,
+    DeviceTypeModel,
+    InterfaceModel,
+    IPAddressModel,
+    LocationModel,
+    ManufacturerModel,
+    PlatformModel,
+    PrefixModel,
+    RoleModel,
+    SoftwareVersionModel,
+)
+from nautobot_ssot.integrations.solarwinds.diffsync.models.nautobot import (
+    NautobotIPAddressToInterfaceModel,
+)
+
+
+class NautobotAdapter(BaseNautobotAdapter):
+    """DiffSync adapter for Nautobot."""
+
+    location = LocationModel
+    platform = PlatformModel
+    role = RoleModel
+    manufacturer = ManufacturerModel
+    device_type = DeviceTypeModel
+    softwareversion = SoftwareVersionModel
+    device = DeviceModel
+    interface = InterfaceModel
+    prefix = PrefixModel
+    ipaddress = IPAddressModel
+    ipassignment = NautobotIPAddressToInterfaceModel
+
+    top_level = [
+        "location",
+        "manufacturer",
+        "platform",
+        "role",
+        "softwareversion",
+        "device",
+        "prefix",
+        "ipaddress",
+        "ipassignment",
+    ]
+
+    def load_param_mac_address(self, parameter_name, database_object):
+        """Custom loader for 'mac_address' parameter."""
+        mac_addr = getattr(database_object, parameter_name)
+        if mac_addr is not None:
+            return str(mac_addr)
+        return mac_addr