In this chapter, we will cover the following recipes:

• JUNOS NETCONF over SSH setup
• Making NETCONF RPC requests and replies
• Using NETCONF to apply configuration changes
• Processing NETCONF using classic Expect/TCL
• Processing NETCONF with Python
• Processing NETCONF with Node.js
• Discovering NETCONF RPCs

The Network Configuration Protocol (NETCONF) standard, defined most recently in RFC 6241, allows a network management application to access a JUNOS OS (or other vendor) network element through the use of a series of Remote Procedure Calls (RPCs) carried over a serialized XML transport.

For programmatic access to JUNOS OS devices, this method is preferable for the use of raw command-line processing, since the data format is structured, precise, and suitable for unambiguous machine reading.

In this chapter, we investigate how to setup NETCONF access to JUNOS OS devices and then look at how to make use of that from common programming platforms.

In this recipe, we'll prepare a JUNOS OS router for interaction using the NETCONF service. We can do this in one of two ways:

• Using NETCONF-over-SSH on dedicated TCP port 830,
• Using NETCONF inline with mainstream SSH communications, on TCP port 22.

We'll set up secure SSH keys and a dedicated username for an automation application. Then we'll configure the systems services hierarchy within the Junos OS for the specific method.

In order to complete this recipe, you need access to a JUNOS OS router, switch, or firewall, and a general-purpose Linux/UNIX management host from which to control it.

The steps to prepare a JUNOS OS router for interaction using NETCONF services are as follows:

1. Verify that SSH is configured on your router by ensuring that you have the following configuration present:

      adamc@router> show configuration system services 
      ssh;

2. Generate SSH keys. Generate a public/private key pair using the SSH utility, ssh-keygen:

      unix$ ssh-keygen -C "JUNOS Automation" -f JUNOS_auto_id_rsa
      Generating public/private rsa key pair.
      Enter file in which to save the key (.ssh/id_rsa):    
      JUNOS_auto_id_rsa
      Enter passphrase (empty for no passphrase): <type nothing here>
      Enter same passphrase again: <again, nothing>
      Your identification has been saved in JUNOS_auto_id_rsa.
      Your public key has been saved in JUNOS_auto_id_rsa.pub.

3. Once completed, verify that you have two new files in your working directory:

4. Configure a dedicated user profile to be used for NETCONF access that makes use of the previously generated key-pair. Apply the .pub file contents to the Junos configuration.

      adamc@router> show configuration system login user auto 
      uid 2001;
      class super-user;
      authentication {
         ssh-rsa "ssh-rsa [ actual key omitted] JUNOS Automation"; ##                
      SECRET-DATA
      }

5. Enable a dedicated NETCONF-over-SSH transport endpoint by configuring the following service:

      adamc@router> show configuration system services 
      ssh;
      netconf {
         ssh;
      }

6. Connect to the NETCONF service to witness the protocol greeting and validate the correct operation:

   unix$ ssh -p 830 -i JUNOS_auto_id_rsa auto@10.0.201.201 -s   
   netconf
   <!-- No zombies were killed during the creation of this user      
   interface -->
   <!-- user auto, class j-super-user -->
   <hello xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
   <capabilities>
   <capability>urn:ietf:params:netconf:base:1.0</capability>
   <capability>urn:ietf:params:netconf:capability:candidate:1.0
   </capability>
   <capability>urn:ietf:params:netconf:capability:confirmed-   
   commit:1.0</capability>
   <capability>urn:ietf:params:netconf:capability:validate:1.0
   </capability>
   <capability>urn:ietf:params:netconf:capability:url:1.0?  
        scheme=http,ftp,file</capability>
   <capability>urn:ietf:params:xml:ns:netconf:base:1.0</capability>
   <capability>urn:ietf:params:xml:ns:netconf:capability:
   candidate:1.0</capability>
   <capability>urn:ietf:params:xml:ns:netconf:capability:confirmed-  
   commit:1.0</capability>
   <capability>urn:ietf:params:xml:ns:netconf:capability:     
   validate:1.0
   </capability>
   <capability>urn:ietf:params:xml:ns:netconf:capability:url:1.0?  
   protocol=http,ftp,file</capability>
   <capability>http://xml.juniper.net/netconf/JUNOS/1.0</capability>
   <capability>http://xml.juniper.net/dmi/system/1.0</capability>
   </capabilities>
   <session-id>35980</session-id>
   </hello>
   ]]>]]>

7. On the same SSH session, issue a test RPC to prove that things are working normally. Enter the highlighted first line of the following text exactly as it is and observe the response:

      <rpc><get-software-information/></rpc>
      <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"   
      xmlns:JUNOS="http://xml.juniper.net/JUNOS/15.1F6/JUNOS">
      <software-information>
      <host-name>router</host-name>
      <product-model>olive</product-model>
      <product-name>olive</product-name>
      <JUNOS-version>15.1F6-S5.6</JUNOS-version>
      <package-information>
      <name>os-kernel</name>
      <comment>JUNOS OS Kernel 64-bit  [    
      20161130.340898_builder_stable_10]</comment>
      </package-information>
      <package-information>
      <name>os-libs</name>
      <comment>JUNOS OS libs [20161130.340898_builder_stable_10]   
      </comment>
      </package-information>
      <package-information> 
      <name>os-runtime</name>
      <comment>JUNOS OS runtime [20161130.340898_builder_stable_10]  
      </comment>
      </package-information>
      […]

In step 1, we verified that the SSH protocol was configured and available in order to access the JUNOS device.

In step 2, we created an SSH public/private key-pair in order to allow any applications that we create to be able to login and authenticate with the JUNOS device in the same way that an ordinary user does. Key-based authentication is preferred over conventional password authentication for this, because it removes the authentication step from the interactive dialog under development.

In step 3, we created a dedicated user profile on the JUNOS device for automation applications and associated it with the public key that we created. Any automation application that makes use of the corresponding private key can be authenticated on the JUNOS OS platform with the public key.

With step 4, we created a NETCONF-over-SSH service endpoint. This isn't technically required, but it can be useful if you would like to treat ordinary user management traffic independently from machine-to-machine programmatic access, and want to enforce such policies via a firewall or similar.

In step 5, we connected to the NETCONF-over-SSH service on port 830 and observed its welcome greeting. We used the -i switch in order to specify the private key that we generated in step 2.

Finally, in step 6, we issued a very basic RPC request to ask the JUNOS OS device for information about its system software. We can see the regularity in the structure of communications between client and NETCONF server. The client's communications consists of a remote procedure call request, enclosed in <rpc></rpc> tags. And the server responds with a document structure enclosed within <rpc-reply></rpc-reply> tags. The actual internal structure of the response depends on the exact RPC called, but the XML format is easier to machine-read than a free-form text interface designed to please a human.

In step 5 and step 6, we saw the guts of the NETCONF protocol dialog occurring. The server said hello to us, and we issued a procedure call which the server duly answered. In actual fact, we were being a little lax in our use of the NETCONF protocol standard there. If you want to speak RFC-compliant NETCONF, it is customary for both the client and the server to issue hello messages that describe their capabilities. The capabilities announced describe concepts over and above some of the base NETCONF principles that are supported by the element, and the manager. In this case, the JUNOS OS server has likely little concern for our client capabilities and takes the IETF mantra of being liberal in acceptance, conservative in communication, to heart.

The other significant point to note is the special sequence of characters used to delimit successive XML messages. We see it at the end of a hello message, and at the end of every RPC response the server answers:

  ]]>]]>

Technically, this framing sequence is actually deprecated within the latest specification of the NETCONF-over-SSH standard, because it was discovered that it can legitimately appear within the XML payload. The JUNOS OS implementation currently makes use of the framing sequence to flag the end of its responses, but if you write software -- as we will -- to read the NETCONF XML stream directly, then it is wise to be aware that this behavior could change in the future.

With NETCONF-over-SSH happily configured on our network of JUNOS OS devices, we can now connect over the network and make RPCs in order to inspect the operational status of the device. Lets look at a couple of examples to learn the fundamentals of how the JUNOS OS XML RPCs work.

Ensure you've completed the JUNOS NETCONF-over-SSH setup recipe previously and have a working JUNOS OS device with a NETCONF interface in place. It doesn't necessarily matter what the configuration of that device is.

The steps for making NETCONF RPC requests and replies are as follows:

1. Connect to the NETCONF-over-SSH server in a similar manner to the previous recipe:

      unix$ ssh -i JUNOS_auto_id_rsa auto@10.0.201.201 netconf

2. Query the system ARP table by connecting to the NETCONF-over-SSH session in a similar manner to the previous recipe and issuing the appropriate RPC:

      <rpc><get-arp-table-information/></rpc>
      <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"    
      xmlns:JUNOS="http://xml.juniper.net/JUNOS/15.1F6/JUNOS">
      <arp-table-information    
      xmlns="http://xml.juniper.net/JUNOS/15.1F6/JUNOS-arp"    
      JUNOS:style="normal">
      <arp-table-entry>
      <mac-address> 
      0a:00:27:00:00:00
      </mac-address>
      <ip-address>
      10.0.201.1
      </ip-address>
      <hostname>
      adamc-mac
      </hostname>
      <interface-name>
      em0.0
      </interface-name>
      <arp-table-entry-flags>
      <none/>
      </arp-table-entry-flags>
      </arp-table-entry>
      </arp-table-information>
      </rpc-reply>
      ]]>]]>

3. Repeat the query, but use the format tag to modify the output to be a plain text:

      <rpc><get-arp-table-information format="text"/></rpc>
      <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"      
      xmlns:JUNOS="http://xml.juniper.net/JUNOS/15.1F6/JUNOS">
      <output> 
      MAC Address Address Name Interface Flags
      0a:00:27:00:00:00 10.0.201.1 adamc-mac em0.0 none
      </output>
      </rpc-reply>
      ]]>]]>

4. Use an option to the ARP table RPC in order to disable the name resolution:

   <rpc><get-arp-table-information format="text"><no-resolve/></get-   
   arp-table-information></rpc>
   <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"    
   xmlns:JUNOS="http://xml.juniper.net/JUNOS/15.1F6/JUNOS">
   <output>
   MAC Address Address Interface Flags
   0a:00:27:00:00:00 10.0.201.1 em0.0 none
   </output>
   </rpc-reply>
   ]]>]]>

5. Query the system routing table and inspect the output:

   <rpc><get-route-information/></rpc>
   [...]

6. Repeat the system routing table query, but apply an argument for a particular destination:

   <rpc><get-route-information>  
   <destination>10.0.201.201</destination></get-route-information>  
   </rpc>
   [...]

In steps 1 and step 2, we connected and issued a simple RPC to query the ARP table from the router. The rather verbose XML response encodes structure that is the machine-readable version of what we see in the CLI when we issue the show arp command. Each data atom is enclosed hierarchically within XML tags indicating its type and any associated properties. This structured output format lends itself particularly well for machine-to-machine automation applications.

In step 3, we issued the same RPC, but requested JUNOS OS to give us the plain text output so that we could compare the difference. In almost all cases, the plain text output seen when we use the format="text" modifier to the RPC is identical to what we would see in the CLI.

In step 4, we see how options to the CLI commands are encoded within the XML RPC format. In this case the show arp no-resolve option is typically used to prevent any name resolution of IP addresses. It's simply an XML subtag to the main <get-arp-table-information> tag.

Steps 5 and 6 go a step further looking at the RPC that implements the show route command. In step 5, we show how arguments are added to the RPC.

Looking at these example RPCs and the XML format within, we can see two clear styles. One pairs together the opening and closing tags in a strict manner, allowing the inclusion of options and arguments. The other allows an abbreviation of an otherwise empty pair of tags by simply using a leading slash. Compare the following two RPCs, which are identically supported by the JUNOS OS XML NETCONF interpreter:

<rpc><get-route-information/></rpc>
<rpc><get-route-information></get-route-information></rpc>

We've seen in the previous recipes how to use some common RPCs to query system state information on our JUNOS OS devices. But how exactly did we discover the cryptic connection between, for example, the CLI command show route and the RPC equivalent <get-route-information>? The JUNOS OS management daemon, mgd, is responsible for speaking the necessary native protocol to the type of client requesting information: either a human operator on the CLI, or a machine interface via XML. It maps the available system calls to both a CLI and an RPC. In this recipe, we'll explore this mapping.

Ensure you have access to a working JUNOS OS device. You don't necessarily need to have completed the previous recipes on setting up NETCONF remote access.

The steps for the recipe are as follows:

1. Log in to the JUNOS OS device using your normal user credentials and choose the operational mode CLI command that you'd like to work with:

      adamc@router> show arp 
      MAC Address Address Name Interface Flags
      0a:00:27:00:00:00 10.0.201.1 adamc-mac em0.0 none

2. Execute the command, but use the pipe modifier in order to query the XML that maps to the corresponding RPC call:

      adamc@router> show arp | display xml rpc 
      <rpc-reply                
      xmlns:JUNOS="http://xml.juniper.net/JUNOS/15.1F6/JUNOS">
      <rpc>
      <get-arp-table-information>
      </get-arp-table-information>
      </rpc>
      <cli>
      <banner></banner>
      </cli>
      </rpc-reply>

3. Repeat the command, but this time use the pipe modifier in order to explore the XML which maps to the response from the RPC call:

      adamc@router> show arp | display xml 
      <rpc-reply    
      xmlns:JUNOS="http://xml.juniper.net/JUNOS/15.1F6/JUNOS">
      <arp-table-information    
      xmlns="http://xml.juniper.net/JUNOS/15.1F6/JUNOS-arp"    
      JUNOS:style="normal">
      <arp-table-entry>
      <mac-address>0a:00:27:00:00:00</mac-address>
      <ip-address>10.0.201.1</ip-address>
      <hostname>adamc-mac</hostname>
      <interface-name>em0.0</interface-name>
      <arp-table-entry-flags>
      <none/>
      </arp-table-entry-flags>
      </arp-table-entry>
      </arp-table-information>
      <cli>
      <banner></banner>
      </cli>
      </rpc-reply>

In step 1, we see the basic command that we're using as a CLI operator.

In step 2, the extra output pipe causes the JUNOS OS management daemon to not actually execute the command, but instead tell us the RPC that it would use if it were executing the command. So, in this case, we can see that it's <get-arp-table-information>, which is the focus of our attention.

In step 3, we get to learn what the likely response from this RPC will be when our automation app makes the RPC call. In this case, the normal tabular format seen by the human is presented to a machine reader, with each of the fields decorated by XML tags. This allows easy and unambiguous interpretation of the response.

Using the JUNOS OS | xml rpc modifier is also particularly useful for understanding how to present complicated arguments. In this case, for example, it's possible to see how we filter the output of the show route command (which would ordinarily be large and unwieldy) for a specific destination and table:

      adamc@router> show route table inet.0 1.0.0.1/32 | display xml         
      rpc 
      <rpc-reply  xmlns:JUNOS=" http://xml.juniper.net/  
       JUNOS/15.1F6/JUNOS">
      <rpc>
      <get-route-information>
      <destination>1.0.0.1/32</destination>
      <table>inet.0</table>
      </get-route-information>
      </rpc>
      <cli>
      <banner></banner>
      </cli>
      </rpc-reply>

Juniper make great efforts to document the JUNOS OS XML API. You can find the latest version of their XML API explorer at https://apps.juniper.net/xmlapi. It provides a browser-based explorer of the configuration tags available and the operational mode RPCs available as in the following screenshot:

As you might expect, NETCONF isn't limited to querying the JUNOS OS device operational status using RPCs. It can also influence the operating state of the device by applying configuration changes. In contrast to other management models like SNMP however, one doesn't manipulate individual data atoms to effect change. Instead, JUNOS OS makes use of the concept of a candidate configuration which is applied to the various software daemons when the candidate is committed. In this respect, NETCONF and the traditional user-based CLI are consistent.

In this recipe, we'll look at the NETCONF directives necessary to make configuration changes. We'll make a simple interface description change, and we'll also look at how to delete configuration stanzas.

Make sure you've got access to a JUNOS OS platform that you can make changes on. Make sure that you've got a working NETCONF-over-SSH capability with the JUNOS OS platform as per the first recipe in this chapter, JUNOS NETCONF-over-SSH setup.

The steps for the recipe are as follows:

1. Familiarize yourself with the XML format used within JUNOS OS to represent configuration data. Generally speaking, the XML representation follows the same hierarchy as the configuration format itself. JUNOS OS itself can help you here. Issue the show configuration | display xml command in order to see a portion of the configuration expressed in XML:

      adamc@router> show configuration interfaces em0.0 | display xml 
      <rpc-reply xmlns:JUNOS="http://xml.juniper.net/JUNOS/
                              15.1F6/JUNOS">
      <configuration JUNOS:commit-seconds="3780" JUNOS:commit-  
       localtime="1970-01-01 01:03:00 UTC" JUNOS:commit-user="adamc">
            <interfaces>
                <interface>
                     <name>em0</name>
                        <unit>
                           <name>0</name>
                           <family>
                               <inet>
                                   <address>
                                       <name>10.0.201.201/24</name>
                                   </address>
                               </inet>
                           </family>
                       </unit>
                  </interface>
               </interfaces>
            </configuration>

2. Connect to the NETCONF-over-SSH server in the usual manner:

      unix$ ssh -i JUNOS_auto_id_rsa auto@10.0.201.201 netconf

3. Use the NETCONF-standard edit-config operation to submit a configuration change to the NETCONF server. In this example, we update the description on the em0.0 interface to something trivial:

     <rpc>
     <edit-config>
       <target>
         <candidate/>
       </target>
       <config>
         <configuration>
           <interfaces>
             <interface>
               <name>em0</name>
               <unit>
                 <name>0</name>
                 <description>Management interface</description>
               </unit>
             </interface>
           </interfaces>
         </configuration>
      </config>
    </edit-config>
    </rpc>

4. Verify that the operation was successful. The <ok/> RPC reply is what we want to see here.
5. Commit the configuration by issuing the commit NETCONF primitive and checking for the <ok/> RPC reply again:

      <rpc><commit/></rpc>
      <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"   
      xmlns:JUNOS="http://xml.juniper.net/JUNOS/15.1F6/JUNOS">
      <ok/>
      </rpc-reply>
      ]]>]]>

6. Apply the same configuration, but delete the description attribute by including the special operation="delete" XML attribute decoration:

      <rpc>
      <edit-config>
        <target>
        <candidate/>
      </target>
      <default-operation>none</default-operation>
      <config>
          <configuration>
            <interfaces>
             <interface>
               <name>em0</name>
               <unit>
                 <name>0</name>
                 <description operation="delete"/>
               </unit>
             </interface>
            </interfaces>
           </configuration>
      </config>
      </edit-config>
     </rpc>

7. Commit the candidate configuration again, and analyze the configuration and system commit log by hand to verify what happened:

   <rpc><commit/></rpc>
   <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"    
   xmlns:JUNOS="http://xml.juniper.net/JUNOS/15.1F6/JUNOS">
   <ok/>
   </rpc-reply>
   ]]>]]>

In step 1, we need to work out what our configuration change looks like in the XML representation that JUNOS OS requires. We can use the CLI to help us with that process.

In step 2, we connect to the NETCONF-over-SSH server in the usual manner.

In step 3, we submit the configuration change that we need, represented in XML, and then in step 4 we look for the server's response. If it isn't the standard <ok/> response, there are a couple of reasons why that might be:

1. The configuration submission contained an error:

    <rpc-error>
    <error-type>protocol</error-type>
    <error-tag>operation-failed</error-tag>
    <error-severity>error</error-severity>
    <error-message>syntax error</error-message>
    <error-info>
    <bad-element>unti</bad-element>
    </error-info>
    </rpc-error>

2. The JUNOS OS configuration database is currently locked by another user:

    <rpc-error>
    <error-type>protocol</error-type>
    <error-tag>lock-denied</error-tag>
    <error-severity>error</error-severity>
    <error-message>
      configuration database locked by:
      adamc terminal pts/0 (pid 19893) on since 1970-01-01 01:09:14    
      UTC, idle 00:03:11
      exclusive [edit]
    </error-message>
    <error-info>
      <session-id>19893</session-id>
    </error-info>
    </rpc-error>

If all is okay, we proceed to the commit operation in step 5. This is the part of the process where the new configuration actually gets applied. JUNOS OS produces the individual instructions for each of the software processes from the configuration file, and then signals each process to re-read the configuration and implement the change to the new state.

This phase can also have errors if the new configuration causes a runtime error. It's really important to deal with this situation because the configuration change will not be removed, so it has the potential to block up future commit operations as well.

Here's the RPC response that we get, for example, if we try to commit an Ethernet sub-interface with zero-host portion:

   <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"    
   xmlns:JUNOS="http://xml.juniper.net/JUNOS/15.1F6/JUNOS">
   <rpc-error>
   <error-type>protocol</error-type>
   <error-tag>operation-failed</error-tag>
   <error-severity>error</error-severity>
   <source-daemon>
      dcd
   </source-daemon>
   <error-path>
      [edit interfaces em0 unit 10 family inet] 
   </error-path>
   <error-info>
   <bad-element>
      address 1.0.10.0/24
   </bad-element>
   </error-info>
   <error-message>
      Cannot assign address 0 on subnet
   </error-message>
   </rpc-error>

In order to ensure that we undo the failed configuration attempt, we can use the discard-changes RPC from NETCONF standard. This will cause the JUNOS OS device to discard any changes in the global candidate configuration that we are working on.

 <rpc><discard-changes/></rpc>

In steps 6 and 7, we undo the change by submitting a new configuration with a delete directive and then committing to that. Configuration deletions are quite simple, but it's important to understand them. There are two notable differences from the configuration addition:

• The default-operation RPC property is set to None.
  This property controls how JUNOS OS applies the supplied configuration with respect to the existing candidate configuration. By default, JUNOS OS merges configuration items, which is typically what we want when we're adding or changing values. But when we're deleting configuration items, we don't want JUNOS OS to accidentally create unnecessary configuration hierarchies.
• The operation property for the item to be deleted is set to Delete.
  This tells JUNOS OS that this particular element should be removed from the configuration.

Don Libes' Expect, extending the ever-flexible Tool Command Language (TCL), forms one of the original ways of automating I/O interaction with terminal-based UNIX processes.

It has been used for numerous applications, from managing the login process on modem dial-up systems, to automating the interaction with network elements in ISP networks in a programmatic way. While this activity of so-called screen-scraping-reading and parsing output -- meant for humans in a machine-compatible way -- can be limited and subject to future-proofing problems, it still represents a significant capability, and sometimes it can be useful to make use of Expect with NETCONF-based network elements.

In this recipe, we explore using a simplistic Expect skeleton program to make RPC calls to our JUNOS OS devices in order to execute commands and extract data.

To complete this recipe, you should have completed the previous recipe, JUNOS NETCONF- over-SSH setup for your device, particularly with respect to establishing SSH key-pairs.

You should ideally make use of Expect 5.45, or a compatible version on your management host. At the time of writing, this version was available in the built-in software package systems of OpenBSD 6.0 and Ubuntu Linux 16.04. Expect is particularly mature and stable however, so if you can't match the exact version, it's unlikely that you'll run into trouble with the example code that we have here.

Our Expect program, netconf.tcl, will be comprised of three main parts, which are as follows:

• Some initialization routines to read the command-line arguments
• Set up of the NETCONF-over-SSH session
• Interaction with the NETCONF-over-SSH session to make an RPC call, and output the response

The steps for the recipe are as follows:

1. Create the interaction procedure first. To do this, create a TCL procedure that accepts a string argument that will represent the command to run:

     proc cmdrpc { cmd } {
        send -- "<rpc><command format=\"text\">[join $cmd]</command>    
    </rpc>\r\n"
      set output ""
      expect {
 
            -re {<error-message>([^<]+)</error-message>} {
              send_error "Command RPC for $cmd caused error:    
       $expect_out(1,string)\r\n"
              return
          }

            -re {<(configuration-)?output[^>]*>} {
               expect {
                  -re {^[^<]+} {
                     append output $expect_out(0,string)
                     exp_continue
                 }
                   -re "</(configuration-)?output>" {}
              }
                regsub -all "&lt;" $output "<" output
                regsub -all "&gt;" $output ">" output
                regsub -all "&amp;" $output "&" output
                return $output
             }

            default {
               send_error "Timeout waiting for RPC [join $cmd]\r\n"
               send_error [
                  concat "\t" [
                      regsub -all {[\r\n]+} $expect_out(buffer)      
                          "\r\n\t"
                     ]
                 ]
                  return
              }
           }
        }

2. Read the environment command-line arguments in order to determine a hostname and a command:

      if { [ llength $argv ] != 2 } {
          send_user "Usage: netconf.tcl hostname command\r\n"
           exit 1
      }
      set hostname [lrange $argv 0 0]
      set command [lrange $argv 1 1]

3. Establish a NETCONF-over-SSH session and call the previously defined interaction procedure to send the RPC and extract the results:

      set DELIMITER {]]>]]>}
       if [ spawn -noecho ssh -p 830 -i JUNOS_auto_id_rsa      
       auto@$hostname -s netconf ] {
       expect {
           $DELIMITER {
           set result [ cmdrpc $command ]
               if {$result ne ""} {
                  send_user $result
               }
          }
            default {
            send_error "SSH protocol error (check       
            authorized_keys?)\r\n"
               exit 1
          }
        }
       } {
        send_error "Unable to start SSH client for connection to   
       $hostname\r\n"
        exit 1
       }
       close
       exit

First of all, the command-line arguments are analyzed to get a hostname and a command to run. Then we use the spawn command to start up a regular SSH client with the necessary parameters to connect to the hostname. Note that we're using the auto username and the key that we explicitly generated in the previous recipes.

The hard work happens in the interaction procedure, cmdrpc. It's comprised of two nested expect loops. First of all, it open the dialog with the NETCONF host by sending the command RPC along with the textual command that we want to execute. The first expect loop runs, which attempts to determine if the RPC was successful or otherwise. If the successful RPC branch is chosen, a second expect loop runs, which accumulates the lines of output in a variable, ready to return. The second expect loop determines the end of the output by looking for the appropriate XML closing tag. Finally the resulting output is scanned to expand some special XML tokens, as per the JUNOS OS specification, and we print the output for the user to see.

Depending on your familiarity with TCL and Expect, you might have a little bit of trouble following the example code. If so, take heart. TCL can seem a little bit daunting because of the quoting and escaping rules that are implemented using braces. In the table, there's a handy phrase - book to compare an example to the typical UNIX shell, which might be a little more widely understood.

In recent years, Python has become one of the de-facto software development languages in the automation and scripting world. Its benefits include an accessible and readable syntax, a just-in-time compilation/interpretation model that allows rapid development cycles, and a batteries included standard library that immediately lends itself to many common situations.

In this recipe, we'll make use of a Python script, netconf.py, to connect to a JUNOS OS device in order to issue CLI-like RPCs, much as in the Expect/TCL example. We'll do this using just the basic standard libraries available out of the box in Python, so there is little fussing about with pip or other package management tools.

In order to complete this recipe, make sure you've got access to a working JUNOS OS device and have completed the JUNOS NETCONF over SSH setup recipe. Additionally, you need a suitable Python development environment. In this case, we made use of macOS X and OpenBSD with Python 2.7.13.

The steps for the following recipe are as follows:

1. Import the necessary standard library modules that we're going to use. In this case, we just need access to basic system functionality, the subprocess module (for managing child processes), and the XML parsing library:

      #!/usr/bin/env python

      import sys
      import subprocess
      import xml.etree.ElementTree as ET

2. Create a Python object class to represent the NETCONF client, making use of the subprocess module in the Python standard library in order to call the underlying operating system's SSH client. Define an appropriate constructor and destructor function as shown:

    class NETCONFClient(object):

      DELIMITER = ']]>]]>\n'

       def __init__(self, hostname):
       self.ssh = subprocess.Popen([
             "/usr/bin/ssh",
             "-q",
             "-i", "JUNOS_auto_id_rsa",
             "-p", "830",
             "-s", 
          hostname,
          "netconf",
           ],
           stdin=subprocess.PIPE,
           stdout=subprocess.PIPE)

       def __del__(self):
          self.ssh.stdin.close()

3. Define a method to read from the NETCONF-over-SSH stream, in a chunked, line-by-line manner, attempting to parse the XML stream.

       def read(self):
         data=""
         for line in iter(self.ssh.stdout.readline,         
       NETCONFClient.DELIMITER):
           if line=='':
           raise IOError("ssh session ended unexpectedly")
       data += line

       return ET.fromstring(data)

4. Define a method to write to the NETCONF-over-SSH stream in order to issue RPCs:

      def cmdrpc(self, cmd):
          e = ET.Element("rpc")
          e.append(ET.Element("command", {'format': "text"}))
          e.find("command").text = cmd;
         self.ssh.stdin.write(ET.tostring(e))
         self.ssh.stdin.write(NETCONFClient.DELIMITER)

5. Write the main code to read the command-line arguments and instantiate the NETCONFClient object:

      if len(sys.argv) < 3:
         print "Usage: netconf.py hostname command"
         sys.exit(1)

         netconf = NETCONFClient("auto@"+str(sys.argv[1]))
         response = netconf.read()
         netconf.cmdrpc(" ".join(sys.argv[2:]))
         response = netconf.read()

         output=response.find(".//{urn:ietf:params:xml:ns:
                   netconf:base:1.0}output")
         config = response.find(".//{urn:ietf:params:xml:ns:
                   netconf:base:1.0}configuration-output")
         error = response.find(".//{urn:ietf:params:xml:ns:
                   netconf:base:1.0}error-message")

6. Output the response:

      if output != None:
        print output.text
      elif config != None:
        print config.text
      elif error != None:
        print error.text
      else:
        print "NETCONF server provided no usable response"

Step 1 sets up the dependent standard library modules. In this case, we use only the well-trodden modules included with the standard Python distribution. The sys module provides access to the command-line environment. The subprocess module provides a flexible way of managing child processes. ElementTree is the Python built-in XML parsing environment.

In step 2, we create a Python new-style class with a constructor and a destructor. The constructor invokes the subprocess module in order to manage a child process consisting of an SSH client. We use the typical options of SSH to influence its behavior:

The destructor attempts to clean up by closing the standard input stream to the SSH client, which will usually result in the SSH client disconnecting from the remote endpoint.

In step 3, we define a method to read data from the SSH client. The data is read line-by-line until we see the special NETCONF delimiter token. When we see that, we know a message has been completed and it is passed to the ElementTree routines for XML decomposition as a Python object.

In step 4, we define the complimenting output method — a function to write a command RPC. The method simply wraps the input parameter — which is the command line to be executed — in the necessary XML decoration in order to invoke the command RPC.

Step 5 is about putting it all together. We read the command-line arguments to determine the hostname and the command to use. Since most commands consist of multiple words, the user is expected to quote the command. For example:

   unix$ ./netconf.py 10.0.201.201 "show route summary"

We call the method to read data from the SSH stream in order to eat the hello message - we've no real need to understand its contents. Then we output a command RPC for the desired command, and call the read method once more in order to receive the response.

As we handle the response from the command RPC, we anticipate receiving one of three types of tag, as shown in the following table:

Finally, in step 6, we print what we've discovered for the user's attention.

Node.js is a popular JavaScript-based language used originally in the server-side web environment, but now common in many application spaces. Its key benefit is its modern JavaScript-dialect allowing object-oriented and prototypal object inheritance models, fused together with Google's efficient V8 JavaScript engine, and an asynchronous, event-based programming framework from the get-go. The asynchronous nature of Node.js makes it ideal for advanced automation and control applications where one needs to communicate with multiple elements at once.

In this recipe, we explore the use of a simple Node.js application acting as a NETCONF client in a similar manner to the previous Python and Expect/TCL applications.

In order to complete this recipe, you should have already completed the JUNOS NETCONF over SSH setup recipe and have a working JUNOS OS NETCONF host. You also need a Node.js installation on the operating system of your choice. For our testing, we used a variety of versions, from v0.10.35 through v6.10.0.

The steps for the recipe are as follows:

1. Firstly, install a viable XML parsing library. Out of the box, Node.js ships with no XML parsing capability within its standard modules, so make use of the popular xml2js library, written by Marek Kubica, and install it using the npm package management tool:

    unix$ npm install xml2js

2. Import the required Node.js modules to operate. In this case, we make use of the child_process module in order to control a child process and the XML parsing module:

    #!/usr/bin/env node

    const util = require("util");
    const child_process = require('child_process');
    const xml2js = require('xml2js');

3. Define some program constants that we can refer to consistently later, including the XML phrase for the command RPC and the invaluable NETCONF delimiter that denotes the space between XML messages:

    const DELIMITER="]]>]]>";

    const xmlRpcCommand = function(command) {
      return [
        "<rpc>\n",
        "<command format=\"text\">\n",
        command,
        "</command>",
        "</rpc>\n",
        DELIMITER,
        "\n"
      ];  
    };

4. Define a convenience utility subroutine for accessing the nested JavaScript object dictionaries, which will be the result of parsing the XML.

  var walk = function(obj, path) {
     var result = obj;
     path.forEach(function (cur, ind, array) {
       if (result) result=result[cur];
     });
     return result;
  }

5. Parse the command-line arguments to determine a target hostname and a command:

  if (process.argv.length!=4) {
     console.warn("Usage: netconf.js user@hostname command\n");
     process.exit(1);
  }
  var hostname = process.argv[2];
  var command = process.argv[3];

6. Start up a child process in order to run the SSH client to connect to the JUNOS OS host:

  var child = child_process.spawn(
     "/usr/bin/ssh", [
       "auto@"+hostname,
       "-q",
       "-p", "830",
       "-i", "JUNOS_auto_id_rsa",
       "-s", "netconf"
     ]
   );

7. Define the important event handlers to deal with the runtime interaction with the SSH session, including handling things like reading data from the SSH session and handling error conditions:

var data="";

child.stderr.on('data', function(chunk) { process.stderr.write(chunk, "utf8"); });
child.stdout.on('data', function(chunk) {
  data+=chunk;
  if ((index=data.indexOf(DELIMITER))!=-1) {
    var xml = data.slice(0, index);
    data = data.slice(index + DELIMITER.length);
    xml2js.parseString(xml, function(err, result) {
      if (err) throw err;
      if (result['hello']) return;
      if (output=walk(result, ['rpc-reply',
                               'output', 0])) {
        console.log(output);
      } else if (config=walk(result, ['rpc-reply',
                        'configuration-information', 0,
                        'configuration-output', 0])) {
        console.log(config);
      } else if (error=walk(result, ['rpc-reply',
                                     'rpc-error', 0,
                                     'error-message', 0])) {
        console.log(error);
      } else {
        console.log("Unexpected empty response");
      }
      child.stdin.end();
    });
  }
});

child.on('error', function(err) { console.log("SSH client error: ", err); })

8. Finally, start the ball rolling by issuing a command RPC for the user-specified CLI command:

xmlRpcCommand(command).forEach(function(cur, ind, array) {               
    child.stdin.write(cur, "utf8")
});

Step 1 sees us prepare the runtime environment by installing a module dependency. Node.js package management system has application dependencies installed in the same directory as the application, rather than polluting system directories. This makes for a more self-contained application, but be aware that the node_modules directory in your application directory is an integral part of your application.

In step 2, we start the source code and we start by pulling in the necessary Node.js modules that we need to reference in this application. We use the child_process module to manage a child SSH session, and we use the xml2js module to do the heavy work of parsing the XML.

Step 3 defines some foundation constants. In this case, we need to use the NETCONF delimiter, as in our other applications, in order to determine where XML messages start and stop. And we also include an XML template for the command RPC that we will call.

In step 4, we create a helper routine. Because the XML parsing process will leave us with complicated JavaScript dictionaries representing each of the tags in the XML document, we want to make a nice, clean and easy syntax to walk an XML structure. Unfortunately, Node.js isn't particularly tolerant to us de-referencing dictionary elements that are non-existent. For example, if we have an object structured like this:

  routers = { 'paris--1': { version: '14.1R6', hardware: 'MX960' },
 'london--1': { version: '15.1F6-S6', hardware: 'MX960' },
 'frankfurt--1': { version: '15.1F6-S6', hardware: 'MX960' } }

We might look to query the software version using syntax like this:

 > routers['paris--1']['version']
 '14.1R6'

Unfortunately, this fails miserably if we try to reference a device that isn't in the dictionary. Node.js throws a TypeError exception, stopping the application in its track:

> routers['amsterdam--1']['version']
TypeError: Cannot read property 'version' of undefined

Instead, we use the walk routine defined in step 4 to conditionally walk a path through a JavaScript object, returning the undefined sentinel value at the earliest failure. This allows us to deal with the error condition on an aggregate basis, rather than checking validity of every element in the path:

    > walk(routers, [ "paris--1", "version" ])
    '14.1R6'
    > walk(routers, [ "amsterdam--1", "version" ])
    undefined

Step 5 sees us use the JavaScript dialect to parse the command-line arguments, and like the previous recipes, we simply look to glean the target hostname and the command to execute.

Then the Node.js magic is put to work in steps 6 and 7. We start off a child process, which involves the operating system forking and executing an SSH client in a similar manner to the previous recipes. But instead of interacting with the SSH client with a series of read/writes, we instead simply define event handlers for what happens in response to certain events, and let the Node.js event loop do the rest.

In our case, we deal with different events, best described in pseudo code in the following table:

Step 8 actually solicits the output from the JUNOS OS device by emitting the RPC command which executes the user's command. When the response is received, the prepared event handlers perform their prescribed activities, which results in the output being printed.