Revolutionising Telecom Efficiency with RED HAT ANSIBLE AUTOMATION

Discover how Red Hat Ansible Automation Platform is transforming network management in the telecommunications industry, offering innovative solutions for enhanced efficiency and security.

In an era where speed and efficiency are paramount, the telecommunications industry faces ongoing challenges in complex network management. Dvloper took on the challenge of providing a robust and innovative solution for a government sector client, addressing the need for efficient management of an extensive and complex network architecture.

Our client, a governmental entity, was struggling with efficiently managing its vast and complex network architecture, which included thousands of geographically dispersed devices. The need for a solution that could automate and optimize processes was clear, and the Red Hat Ansible Automation Platform was chosen to meet this need.

Client Infrastructure:

Thousands of network devices

               -         Hub and spoke network model

               -         Branches: Routers, Firewalls, Branch Switches

               -         Data Centers: Routers, Firewalls,  Datacenter Switches

               -         Aggregators

Different OSs from multiple vendors

Issues & Complexities faced by client:

Scalability Issues

               -         Resource Intensiveness

Human Error

               -         Configuration Discrepancies

               -         Time-Consuming Updates

Operational Downtime

               -         Service Disruptions

Lack of Centralised Control

               -         Limited Visibility

               -         Difficulty in Tracking Changes

Implementation Strategy:

Objective 1: Install the airgapped Ansible Automation Platform

We recommend that you create a new VM from where you will perform the AAP installation. Follow Red Hat documentation and decide on a type of deployment that best suits your environment. ( Automation Controllers, Execution Nodes, Private Automation Hub, EDA )

Red Hat provides detailed documentation on how to install the AAP platform, however we faced quite a big challenge. The challenge was that we had to perform the installation in an air gapped environment meaning that no internet connection was available.  For the deployment to succeed we had to perform the bellow prerequisites.

Create local repository on RHEL 9

Red Hat maintains an offline bundler installer for AAP, however you still need to have all the Auxiliary needed Packages. The simples way we found to overcome this was to mount the offline installer Rhel ISO on every node as a folder and to configure yum to retrieve packages from it:

●      Check that the RHEL iso is connected

lsblk

●      Mount the RHEL iso to /media directory:

sudo mount /dev/sr0 /media

●      Create a directory to copy all the files from the ISO mounted on /media directory

mkdir /local-repo
cp -rfv /media /local-repo/

●      Create YUM repository file at this path as root user: /etc/yum.repos.d

sudo su -
touch /etc/yum.repos.d/local.repo && cat << EOF >> /etc/yum.repos.d/local.repo
[BaseOS]
name=baseosforrhel
baseurl=file:///local-repo/media/BaseOS
enabled=1
gpgcheck=0
metadata_expire=-1
[AppStream]
name=appstreamforrhel
baseurl=file:///local-repo/media/AppStream
enabled=1
gpgcheck=0
metadata_expire=-1
EOF

●      List the newly created local repo

yum repolist

●      Umount the RHEL iso from /media directory and eject the disk:

umount /media

Other prerequisites

●    Append to /etc/hosts  files on every server the FQDN and IP addresses of every node including the node that from where you will perform the deployment.

●    Generate a ssh-key on the deployment VM and copy it on all the nodes to allow Ansible to connect to the AAP nodes with the ssh-key (Generate the key as a 4096b RSA key and use ssh-copy-id from the deployment node to upload it to the AAP nodes)

●    Enable passwordless sudo on all nodes for the deployment user (revert after install)

●    Enable root login on all nodes (revert after install)

●    Generate the certificates for the nodes. You will pass this certificates in the AAP deployment inventory file. You need to generate the CA, intermediate CA and server certificates and keys.

●    Disable the official RHEL repository: subscription-manager config --rhsm.manage_repos=0

Prepare Inventory file

 In our case we decided to go with a single Automation Controller, three Execution Nodes and one Private Automation Hub with the database being deployed on the Automation Controller.

 Example inventory file:

[automationcontroller]
automation-controller-FQDN ansible_host=x.x.x.x peers=execution-node-1-FQDN,execution-node-2-FQDN routable_hostname=automation-controller-FQDN
 [automationcontroller:vars]
node_type=control
 [execution_nodes]
execution-node-1-FQDN ansible_host=x.x.x.x routable_hostname=execution-node-1-FQDN
execution-node-FQDN ansible_host=x.x.x.x routable_hostname=execution-node-2-FQDN
[execution_nodes:vars]
node_type=execution 
[automationhub]
private-automation-FQDN ansible_host=x.x.x.x routable_hostname=private-automation-hub-FQDN
[all:vars]
ansible_connection=ssh
ansible_become=true
ansible_user='Deployment User'
host_key_checking=False
pg_host=''
pg_port=5432
pg_database='awx'
pg_username='awx'
pg_password='pg-password'
pg_sslmode='prefer'  # set to 'verify-full' for client-side enforced SSL
admin_password='Automation Controller Portal admin pass'
automationhub_admin_password='Automation Hub Portal admin pass'
automationhub_pg_host='automation-controller-FQDN'
automationhub_pg_port=5432
automationhub_pg_database='automationhub'
automationhub_pg_username='automationhub'
automationhub_pg_password='automation hub pg password'
automationhub_pg_sslmode='prefer'
# If set, this will install a custom CA certificate to the system trust store.
custom_ca_cert=/home/rhel/ansible_automation_platform/ansible-automation-platform-setup-bundle-2.4-1.2-x86_64/privateca/rootCACert.pem
# Certificate and key to install in nginx for the web UI and API
web_server_ssl_cert=/home/rhel/ansible_automation_platform/ansible-automation-platform-setup-bundle-2.4-1.2-x86_64/privateca/tower2.crt
web_server_ssl_key=/home/rhel/ansible_automation_platform/ansible-automation-platform-setup-bundle-2.4-1.2-x86_64/privateca/tower.key
 # Certificate and key to install in Automation Hub node
automationhub_ssl_cert=/home/rhel/ansible_automation_platform/ansible-automation-platform-setup-bundle-2.4-1.2-x86_64/privateca/hub2.crt
automationhub_ssl_key=/home/rhel/ansible_automation_platform/ansible-automation-platform-setup-bundle-2.4-1.2-x86_64/privateca/hub.key

 Run the installation script

 ~/ansible_automation_platform/ansible-automation-platform-setup-bundle-2.4-1.2-x86_64/setup.sh

Congratulations you are on your way to successfully deploy the AAP platform in an air gapped environment!

Next step would be to create your first project in AAP. The beautiful thing is that  AAP can easily integrate with a Git Repository software like Gitea, GitLab, GitHub to  get all the benefits from such a tool.

Benefits:

• Version Control and History Tracking:

               -         Code Consistency

               -         Traceability

• Collaboration and Teamwork:

               -         Concurrent Development

               -         Code Review and Approval

To see the how to install Gitea in an air gapped environment follow this Blog.

Extra tip: Legacy devices or missing modules.

Sometimes you may encounter legacy devices which have very old versions of SSH daemon which can lead to connectivity issues to those devices. No worries! AAP allows you to create custom Execution Environments that take care of these issues. Follow the official Red Hat AAP documentation to learn more about Execution Environments.

In meantime here is an example of how to create your execution environment starting from the default AAP execution environment:

●      Create a venv for ansible builder

python3 -m venv ee-venv
source ee-venv/bin/activate

●      Install ansible builder package

 pip3 install ansible-builder

●      Create execution-environment.yml file

 cat <<EOT >> execution-environment.yml
---
version: 1
build_arg_defaults:
  EE_BASE_IMAGE: 'registry.redhat.io/ansible-automation-platform-24/ee-supported-rhel8:latest'
dependencies:
  galaxy: requirements.yml
EOT
●      Create requirements.yml file
 cat <<EOT >> requirements.yml
---
collections:
  - name: fortinet.fortios
  - name: fortinet.fortiswitch
  - name: fortinet.fortimanager
  - name: fortinet.fortianalyzer
  - name: fortinet.fortiflexvm
EOT

●      Build the EE custom Docker image

ansible-builder build -v3 -t custom-ee

●      From the node where you have the image, login to the private automation hub:

[rhel@utility ~]$ podman image ls
REPOSITORY                 TAG         IMAGE ID      CREATED       SIZE
localhost/custom-ee-rhel8  latest      c8d32d51316f  30 hours ago  2.1 GB
 [rhel@utility ~]$ podman login -u=admin -p=rhel private-automation-hub-1.dvloper.io
Login Succeeded!

Note: You may encounter the following error when logging in.

[rhel@utility ~]$ podman login -u=admin -p=rhel private-automation-hub-1.dvloper.io

Error: authenticating creds for "private-automation-hub-1.dvloper.io": pinging container registry private-automation-hub-1.dvloper.io: Get "https://private-automation-hub-1.dvloper.io/v2/": x509: certificate signed by unknown authority

Resolution:  Edit the /etc/containers/registries.conf file.

  Add the following line:

[registries.insecure]
registries = ['private-automation-hub-1.dvloper.io']

●      Upload the image to the Private automation hub

Tag the new ee:

$ podman tag localhost/custom-ee-rhel8:latest private-automation-hub-1.dvloper.io/custom-ee-rhel8:latest

Push the new ee:

$ podman push private-automation-hub-1.dvloper.io/custom-ee-rhel8

Now you can use your custom Execution Environment image after you share it with the Automation Controller.

Objective 2: Automate tasks with AAP

Goals:

        -   Create Network automation playbooks:

• Initial Configuration: Hostname, SSH, NTP, TACACS,

§  SNMP, DNS, Banner, etc.

• VLAN and VRF configuration
• Interface Configuration
• Static and Dynamic Routing

§  protocols configuration: Static routes/ BGP/OSPF

• Site-to-site IPsec VPN tunnels
• ACLs / Firewall policies
• Zone configuration
• Automated Roll Back 

The goal is to reach to a state of automation in which you can just simply specify the parameters like in the examples below:

Achieved Results:

The Ansible implementation brought significant benefits, including:

●      Process Optimization: The time required for configurations and updates was reduced by 50%, allowing teams to focus on higher value-added tasks.

●      Enhanced Security: Automation ensured consistent implementation of security policies, reducing the risk of human errors.

●      Improved Incident Resolution Efficiency: The response time for IT incidents decreased by 40%, increasing the stability of the IT infrastructure.

●      Operational Cost Reduction: Operational costs were reduced by approximately 30% through minimizing manual management.

The implementation of the Red Hat Ansible Automation Platform represented a significant step in our client's digital transformation, bringing increased operational efficiency and enhanced security. These results underscore the added value we provide through our innovative and customized IT solutions.

Are you ready to transform your company's IT infrastructure management? Contact our team of experts to discover how we can assist you!

Annex - RedHat Ansible Automation Platform Technical Guide:

Ansible automation platform is designed to help you strengthen three fundamental pillars for a Resilient and Efficient network operation via the Infrastructure as Code concept:

Configuration Management

 Platform agnostic configuration management to standardize and enforce best-practices.

Infrastructure Awareness

Track network resources through facts gathering, to perform preventive maintenance, reducing outage risks and costs of unnecessary hardware-refresh.

Network Validation

Examine operational state to check network connectivity and protocols and to enhance operational workflows to help measure network intent.