Chapter 3. YANG Explained
This chapter covers
What a YANG module looks like and what it contains
Building a simple YANG module, then extending it
How to describe tabular data in YANG
Defining actions, RPCs, and notifications
The fundamental difference between configuration and operational data
Why and how to ensure data is valid using constraints of many different types
Modeling data that is only relevant sometimes
Properly navigating with YANG pointers, using XPath
Augmenting YANG modules, evolving them over time
Network Management Datastore Architecture
Introduction
This chapter takes you on a journey to build a real, usable YANG model. This is done in stages with different themes, building up a more and more comprehensive model. Each stage is available as a hands-on project you can clone from GitHub, and then build, run, and play with as you please. Instructions for how to obtain the necessary free tools are found within the project README file at https://github.com/janlindblad/bookzone.
Describe Your World of Data
Consider the following situation: A fictitious bookstore chain called BookZone decided to define a proper, consistent interface for each store so that client applications can browse the inventory of each store and the central management applications can add new titles and authors to the affiliate stores.
Table 3-1 shows a data structure with the sort of information BookZone wants to keep track of, i.e. the title, ISBN, author and price of each book. ISBN stands for International Standard Book Number. It is used by publishers and book stores to uniquely identify a particular book title, including its edition and variation (for example, hardcover or paperback). The table has some sample data filled in. Such data is called instance data. In order to get organized, you need to create a schema for this data so you can describe it to your peers with precision rather than by example. Examples are great for general understanding, but they are never precise.
Table 3-1 Tabular Representation of the Book Catalog Using Example Data
title |
isbn |
author |
price |
|---|---|---|---|
The Neverending Story |
9780140386332 |
Michael Ende |
8.50 |
What We Think About When We Try Not To Think About Global Warming: Toward a New Psychology of Climate Action |
9781603585835 |
Per Espen Stoknes |
16.00 |
The Hitchhiker’s Guide to the Galaxy |
0330258648 |
Douglas Adams |
22.00 |
The Art of War |
160459893X |
Sun Tzu |
12.75 |
I Am Malala: The Girl Who Stood Up for Education and Was Shot by the Taliban |
9780297870913 |
Malala Yousafzai |
19.50 |
A description of the YANG structure of this table is provided shortly. As time goes by, the instance data (that is, the particular entries in this table) may come and go, but the YANG structure with these four columns remains.
To define a basic book catalog schema in YANG, refer to Example 3-1.
Example 3-1 YANG Schema Representation of the Book Catalog
container books {
list book {
key title;
leaf title {
type string;
}
leaf isbn {
type string;
mandatory true;
}
leaf author {
type string;
}
leaf price {
type decimal64 {
fraction-digits 2;
}
units sim-dollar;
}
}
}
Notice that each line in YANG consists of a keyword (container, list, key, leaf, type, mandatory) followed by a name (books, book, title, isbn, or author) or a value with predefined meaning (string, true). Each line is followed either by a semicolon (;) or by a block surrounded by curly braces ({...}).
A YANG container is suitable when you have a collection of information elements that belong together. By placing the container books at the top level like this, it is easy for the user to find everything related to books. Authors are added later, and they go in a different container, obviously.
A YANG list works like a container, placing related things together—except that with a list, you can have many instances of what is inside the list. Inside the list book, there will be many titles. Think of a YANG list as a table with columns.
By placing a YANG leaf inside the list four times, you get a table with four columns. Each leaf has a type statement inside that defines what kind of data this particular leaf can hold. In this case, three leafs are strings, and one is a number with two digits after the decimal point. The leaf isbn has been marked mandatory true. This means that it must have a value. By default, all leafs are optional, unless they are marked as keys or as mandatory. In Example 3-1, both isbn and title are mandatory, while author and price are optional.
Think of the whole structure as a tree, with the container books as the trunk, the list book as the branching point of four branches, and then the four leaves: title, isbn, author, and price.
The key title statement in Example 3-1 means the title column is the key column of this list. The key column is what identifies which entry is which in the list. For example, if you wanted to update the author value (“Sun Tzu”) of the book The Art of War, so that it also includes his name written with Chinese characters, you would have to use the key column to specify which of the books in the list you wanted to update. If you are familiar with relational databases, the key concept in YANG corresponds with the primary set of keys in a relational table.
You could say the following: Update /books/book[title=‘The Art of War’]/author to be “Sun Tzu (孫子)”. This notation with slashes and brackets is called XPath; it is the language used to navigate the instance data in a database that uses the YANG model as its schema. XPath is explained in some more detail later in this chapter, but just to step through the reference at the beginning of this paragraph, you read the XPath expression like this:
The slash (/) at the beginning means go to the root of the YANG model. Next, books/book means go into the container books, then into the list book. The square brackets imply filtering among the instances. There are five instances in this example. The filter matches only the one that has the title value equal to the given string. When it’s found, navigate to the leaf author. This way, the XPath expression points out a single element. That element gets a new value, which includes some Chinese characters.
Let’s discuss the selection of the key for the list for a moment. In this list, the leaf title was selected to be the key. Another viable candidate could have been isbn, since that, too, could be used to uniquely identify a book. Alternatively, inventing and adding a completely meaningless random value, such as a universally unique identifier (UUID) or a plain number, would work. There’s no right or wrong choice, but the choice decides how a user would interact with the system.
By choosing title to be the key, you have a meaningful identifier to use when thinking about the data. Had you chosen isbn as the key, the update to the author name would look like this: Update /books/book[isbn=‘160459893X’]/author to be “Sun Tzu (孫子)”. Clearly, this less meaningful key increases the chance of mistakes going through undetected. This is a general philosophy in YANG modeling: use meaningful keys, when possible, and allow the user to use any string as an identifier rather than, for example, only numbers. If there was a strong requirement to support multiple books with identical titles, obviously title would not work as a key. In this case, maybe isbn would be the best remaining option. It has at least some real-world identification value.
Before the YANG in Example 3-1 is put to actual use, it needs a boilerplate header. The header uniquely identifies the module and states what revision date it has. The complete, first version of the module is shown in Example 3-2.
Example 3-2 The Complete bookzone-example.yang Module for a Basic Catalog of Books
module bookzone-example {
yang-version 1.1;
namespace 'http://example.com/ns/bookzone';
prefix bz;
revision 2018-01-01 {
description "Initial revision. A catalog of books.";
}
container books {
list book {
key title;
leaf title {
type string;
}
leaf isbn {
type string;
mandatory true;
}
leaf author {
type string;
}
leaf price {
type decimal64 {
fraction-digits 2;
}
units sim-dollar;
}
}
}
}
Each YANG module must start with the keyword module, followed by the module name. The filename of the module must match this name, plus an extension of .yang. The next line declares that the module is written in YANG 1.1, as opposed to YANG 1.0, or any other future version. In this little example, everything would still work fine if you specified yang-version 1.0, but in later updates to this module there are a few things that use YANG 1.1 features. Let’s use yang-version 1.1. That should also be the default when you are writing new modules at this time; there is no particular benefit to use YANG 1.0.
Note
More information about the differences between YANG 1.1 and YANG 1.0 is available in RFC 7950 Section 1.1.
Each YANG module must have a world-unique namespace name, defined by the namespace statement. No two YANG modules in the world should have the same namespace string. It’s up to you, as a YANG author, to ensure that. The key thing with the namespace is that it should be world unique, so the recommendation is that you take your organization’s URL and then add something to ensure it will be unique within your organization. That’s why, in Example 3-2, the namespace string looks like a URL. If you paste that into your browser, however, you’ll get a 404 (page not found) error.
Sometimes you see namespaces beginning with urn:. Such namespaces are used by IETF, for example. Namespaces starting with urn: are supposed to be registered with the Internet Assigned Numbers Authority (IANA). Many organizations mistakenly pick such namespace strings without registering them, however. Do not do that, or you risk pointless trouble.
Since namespace strings must be world unique, they also tend to be rather long, so there is something called a prefix, which is essentially an abbreviation of the module name. In theory, prefixes do not need to be unique. In practice, however, users (operators, programmers, and DevOps engineers) as well as many tools tend to be rather confused when the same prefix is used by several YANG modules. Be sure to pick prefixes that are likely to be unique, especially within the set of modules you author. Example 3-2 uses “bz” as an abbreviation for bookzone. Ideally, a little longer prefix would be good to improve the chances of uniqueness. If you are designing a YANG module, don’t pick an obvious prefix name like if, snmp, bgp, or aaa, because many other module authors have already used them. YANG module readers and users easily get confused when there are several different modules with the same prefix from different vendors.
Next, there is a revision statement with a date. These statements are not strictly required, but it is certainly a good idea to add one every time a new version of the YANG module is “published.” A YANG module is typically considered published when it becomes accessible to people or systems outside the team responsible for it. Anyone will agree who has had to sort out a situation with different versions of programming interfaces with no easy way to determine whether they are the same and, if not, which one is the newer one.
Describe Your Data with Precision
Keeping track of book inventory is certainly key for a bookstore. Before you start doing business, however, you need to keep track of users (who will pay you) and authors (whom you have to pay).
Example 3-3 adds a few lines of YANG for basic user management. Each user has a system-unique user ID (user-id) and a screen name that may occasionally be the same for multiple users. The type string in YANG allows the full UTF-8 character set, so nothing special is required to allow German or Japanese users on the system.
Example 3-3 Simple List of Users with user-id as the Key and a Screen Name
container users {
list user {
key user-id;
leaf user-id {
type string;
}
leaf name {
type string;
}
}
}
Then there are the authors. In Example 3-4 it was decided that all authors be addressed by their full name, and not use any sort of user IDs or numbers, since that has caused confusion in the past. Model creators should decide how to address the items in a list, and this time the full name was chosen. As a consequence, you have to ensure that the full name used for authors contains middle names and suffixes like “Jr.” and “III” so that all authors are uniquely identified by name. Additionally, each author must have a payment account number with the finance department. A single unsigned integer works. Since you know the finance department numbers its accounts starting at 1001, it is reasonable to say so in the model and catch potential errors. The keyword max in the range simply means the highest possible number the type can represent.
Example 3-4 List of Authors with name as the Key and an account-id
container authors {
list author {
key name;
leaf name {
type string;
}
leaf account-id {
type uint32 {
range 1001..max;
}
}
}
}
Since containers users and authors with their lists go outside the books/book container and list, you’re forming a small forest in the YANG module, with three tree-like structures. In order to get an overview, you can use the so-called tree representation to display the YANG module in a compact way, as shown in Example 3-5.
Example 3-5 YANG Tree Representation of the Book Catalog
module: bookzone-example
+--rw authors
| +--rw author* [name]
| +--rw name string
| +--rw account-id? uint32
+--rw books
| +--rw book* [title]
| +--rw title string
| +--rw isbn string
| +--rw author? string
| +--rw price? decimal64
+--rw users
+--rw user* [user-id]
+--rw user-id string
+--rw name? string
The tools and specification (RFC 8340) behind these tree diagrams are discussed further in Chapter 7, “Automation Is as Good as the Data Models, Their Related Metadata, and the Tools: For the Network Architecture and Operator.” You have a list of authors now. There is a leaf in the book list that names the author. Remember? The following snippet depicts the leaf author as a plain string:
leaf author {
type string;
}
This string should, of course, match the name of one of the authors in the list author; otherwise, how would you know which author to pay? The construct in the previous snippet allows mistakes. In YANG, it is easy to close that loophole; just make this a leaf reference instead, or leafref, as it’s called in YANG.
A leafref is a pointer to a column in some YANG list somewhere, and any value that happens to exist in that key column is a valid value for the pointing leaf. In this case, the leafref version of the leaf author is shown in Example 3-6.
Example 3-6 Leaf author from Example 3-2 as a leafref
leaf author {
type leafref {
path /authors/author/name;
}
}
Notice that the leafref contains a path pointing out which YANG list and which key column in that list contains the valid values for the author value. The expression in the path statement is simplified XPath, meaning not all XPath magic can be used here. The path starts at the root of the YANG tree (/), then goes into container authors and list author. Finally, the path points to the name leaf in the list.
Any /authors/author/name leaf (a string) is a valid value for the /books/book/author leafref, which is therefore also a string.
If the /books/book/author leaf changes to something that does not exist in any /authors/author/name, validation of the transaction containing the change fails, since the reference no longer points to a valid instance. It isn’t an error to change the /books/book/author leaf to a value that is not listed in /authors/author/name at the time you are typing this in, as long as you ensure the name is added to /authors/author/name before the transaction is committed.
Another thing you want to add to books is information about which language they are written in, so that client applications can search and display books that may be more relevant to users.
It would be easy to simply add a leaf called language with a string value for the language. However, in YANG, you normally want to be specific about the allowed values when possible. Here, an enumeration of possible language options is fitting. You could enumerate all the languages right inside the leaf, but since this language choice may be relevant in other places in the model, you should define a reusable type for languages. The leaf language with the user-defined type language-type is shown in the following snippet and looks like this in the /books/book list:
leaf language {
type language-type;
}
And somewhere in the YANG module you need to define the language type, as shown in Example 3-7.
Example 3-7 Type Definition of the language-type
typedef language-type {
type enumeration {
enum arabic;
enum chinese;
enum english;
enum french;
enum moroccan-arabic;
enum swahili;
enum swedish;
// List not exhaustive in order to save space
}
description
"Primary language the book consumer needs to master "+
"in order to appreciate the book's content";
}
The convention in YANG is to have all identifiers in all lowercase (with dashes when necessary) for anything an operator might type. Note that this convention applies to enumerations as well. This makes it easy for operators to type the names, if need be. YANG is case-sensitive, so enum ARABIC and enum arabic are two distinct values.
Similarly, the ISBN for the book is currently a string, but as you know, ISBNs have a certain format. Actually, there are (essentially) two variants of ISBNs: one is 10 digits and the other, more modern one is 13 digits, which is adapted to the barcode system. Because you need to handle some older literature, the YANG module must support both formats. The following snippet shows how to sharpen the leaf isbn type to allow precisely the format needed. Leaf isbn is lowercase; remember the YANG convention to use all lowercase identifiers.
First, change the type from type string to a type you define yourself:
leaf isbn {
type ISBN-10-or-13;
}
From here, you need to define this type. By convention, type definitions are placed near the top of the module, as shown in Example 3-8. It’s fine if type names contain uppercase letters, as type names are not typed by operators anyway.
Example 3-8 Type Definition of the User-Defined Type ISBN-10-or-13
typedef ISBN-10-or-13 {
type union {
type string {
length 10;
pattern '[0-9]{9}[0-9X]';
}
type string {
length 13;
pattern '97[89][0-9]{10}';
}
}
description
"The International Standard Book Number (ISBN) is a unique
numeric commercial book identifier.
An ISBN is assigned to each edition and variation (except
reprintings) of a book. [source: wikipedia]";
reference
"https://en.wikipedia.org/wiki/International_Standard_Book_Number";
}
The union YANG construct allows multiple types—any type listed within the construct. In this case, you have two member types: two strings, each of a specific length (10 and 13 characters, respectively). On top of this requirement, the 10-character variant must consist of the digits 0–9 repeated nine times, and then a last check digit (either 0–9 or the letter X). This is simply how 10-digit ISBNs are defined. The 13-character variant always starts with a 9 followed by a 7. The third digit is either 8 or 9. Then follows 10 digits (0–9). This reflects how the ISBN was fit into the barcode system, as if all books were produced in a single country (called bookland). The first few digits in barcodes normally indicate the country of origin.
Since ISBNs are defined by a standard and a standards body, it’s nice to give this type a suitable description. The YANG keyword reference is used to help the reader to find more information on the topic—usually the name of some specification, or as in this case, a link to the Web.
By using specific type definitions (for example, pattern, union, and range), you make the YANG contract clear, and systems can validate the input and output automatically.
Separate Your Data into Categories
Another aspect to consider when it comes to ISBNs is that there is not one ISBN per title; instead, there is one ISBN per title and delivery format. If a given title comes in paperback, hardcover, and EPUB, each one has a different ISBN. With the current model, you need to have separate entries for every book, depending on which format it is in. For your purposes, this is not ideal. Instead, make a list of formats and ISBNs inside the list of books, as shown in Table 3-2.
Table 3-2 Tabular Representation of the List Format with ISBN Information Inserted into List See Example 3-9
title |
author |
formats |
||
|---|---|---|---|---|
The Neverending Story |
Michael Ende |
isbn |
format |
price |
9780140386332 |
paperback |
8.50 |
||
9781452656304 |
mp3 |
29.95 |
||
What We Think About When We Try Not To Think About Global Warming: Toward a New Psychology of Climate Action |
Per Espen Stoknes |
isbn |
format |
price |
9781603585835 |
paperback |
16.00 |
||
The Hitchhiker’s Guide to the Galaxy |
Douglas Adams |
isbn |
format |
price |
0330258648 |
paperback |
22.00 |
||
9781400052929 |
hardcover |
31.50 |
||
The Art of War |
Sun Tzu |
isbn |
format |
price |
160459893X |
paperback |
12.75 |
||
I Am Malala: The Girl Who Stood Up for Education and Was Shot by the Taliban |
Malala Yousafzai |
isbn |
format |
price |
9780297870913 |
hardcover |
19.50 |
||
Having lists inside lists is perfectly normal in YANG. Some people who spent a lot of time with relational databases and modeling find their heads spinning at this point. Actually, there is nothing particularly strange about having lists inside lists. Your computer’s file system has directories inside directories, and nobody finds that strange at all.
This reflects how typical network elements have their command-line interfaces (CLIs) organized. YANG was designed to be able to reflect this natural way of modeling device management data. In fact, one of the reasons Simple Network Management Protocol (SNMP) was considered so hard to use was just this: Living in the relational world, where tables inside tables are not possible, made the SNMP model look very different from how most people looked at and worked with the data in the network elements. There was always a high cost of translation between users’ minds (living in the CLI world) and the relational SNMP tables.
The structure of the model with a list inside a list is shown in Example 3-9.
Example 3-9 List Format Added Inside the List book
container books {
list book {
key title;
leaf title {
type string;
}
...
list format {
key isbn;
leaf isbn {
type ISBN-10-or-13;
}
leaf format-id {
...
}
}
What about the leaf format-id? You could make that a string. This allows for all possible formats, current and future. However, it will inevitably lead to some misspelled or ambiguous entries. An enumeration is clearly better than a string here, since that restricts the user to a predetermined set of well-defined values. The downside with enumerations is that they are hard to expand over time, and they do not cope well with variants.
In the language enumeration there is both Arabic and Moroccan-Arabic. The latter is a variant of the former, but in the enumeration the values are distinct. If you have some YANG expressions that depend on whether a language is Arabic or not, it would have to test against both arabic and moroccan-arabic. And, if you add another Arabic variant, the expression would need to be updated to take each new variant into account.
This concept of some enumeration values belonging together is common in networking, and in the world at large. Just think of how many interface types there are, and how many variants of Ethernet there are now, all the way from 10-Base-T to 1000GE. In YANG, these enumerations with kind-of relations are modeled using the YANG keyword identity. If you are familiar with object-oriented design principles, this is referred to as subclassing in that context.
Here are the book formats that BookZone chose to use identities to describe. See how each identity has a base. Identity paperback has a base of paper, which means it is a “kind-of” paper book. Identity paper, in turn, has a base of format-idty, which means paper is a “kind-of” book delivery format. The suffix -idty is just an abbreviation added here to the each of the root identities to make it easier to remember it is an identity, as shown in Example 3-10.
Example 3-10 YANG Identities Forming a Tree of Is-a-Kind-Of Relations
identity format-idty {
description "Root identity for all book formats";
}
identity paper {
base format-idty;
description "Physical book printed on paper";
}
identity audio-cd {
base format-idty;
description "Audiobook delivered as Compact Disc";
}
identity file-idty {
base format-idty;
description "Book delivered as a file";
}
identity paperback {
base paper;
description "Physical book with soft covers";
}
identity hardcover {
base paper;
description "Physical book with hard covers";
}
identity mp3 {
base file-idty;
description "Audiobook delivered as MP3 file";
}
identity pdf {
base file-idty;
description "Digital book delivered as PDF file";
}
identity epub {
base file-idty;
description "Digital book delivered as EPUB file";
}
The point with recording these relations becomes apparent in later sections in this chapter.
Now that you have a number of formats defined, return to leaf format-id. By you specifying type identityref and a root identity, this leaf can take the value of any identity that directly or indirectly is based on this root identity. As shown in later sections, additional identities may also be defined in other modules. When they are, they immediately become valid values for the format-id leaf. Example 3-11 shows how leaf format-id is typed as a (mandatory) identityref with a base format-idty.
Example 3-11 Leaf format-id with Type identityref
leaf format-id {
mandatory true;
type identityref {
base format-idty;
}
}
By you specifying type identityref base format-idty, all the book formats previously listed immediately become valid values for this leaf, because they are based, directly or indirectly, on format-idty. Leaf format-id was made mandatory since each ISBN is for a given format. You do not want to allow listing a book with an ISBN without saying which format you are talking about.
Before you go ahead and publish all of these additions, take a moment to write a new revision statement. The revision statement should describe what is new since the last revision, and each new revision should be placed before earlier revision statements.
With that addition, the entire module looks like Example 3-12.
Example 3-12 Complete YANG Module at Revision 2018-01-02
module bookzone-example {
yang-version 1.1;
namespace 'http://example.com/ns/bookzone';
prefix bz;
import ietf-yang-types {
prefix yang;
}
revision 2018-01-02 {
description
"Added book formats, authors and users, see
/books/book/format
/authors
/users";
}
revision 2018-01-01 {
description "Initial revision. A catalog of books.";
}
typedef language-type {
type enumeration {
enum arabic;
enum chinese;
enum english;
enum french;
enum moroccan-arabic;
enum swahili;
enum swedish;
// List not exhaustive in order to save space
}
description
"Primary language the book consumer needs to master "+
"in order to appreciate the book's content";
}
identity format-idty {
description "Root identity for all book formats";
}
identity paper {
base format-idty;
description "Physical book printed on paper";
}
identity audio-cd {
base format-idty;
description "Audiobook delivered as Compact Disc";
}
identity file-idty {
base format-idty;
description "Book delivered as a file";
}
identity paperback {
base paper;
description "Physical book with soft covers";
}
identity hardcover {
base paper;
description "Physical book with hard covers";
}
identity mp3 {
base file-idty;
description "Audiobook delivered as MP3 file";
}
identity pdf {
base file-idty;
description "Digital book delivered as PDF file";
}
identity epub {
base file-idty;
description "Digital book delivered as EPUB file";
}
typedef ISBN-10-or-13 {
type union {
type string {
length 10;
pattern '[0-9]{9}[0-9X]';
}
type string {
length 13;
pattern '97[89][0-9]{10}';
}
}
description
"The International Standard Book Number (ISBN) is a unique
numeric commercial book identifier.
An ISBN is assigned to each edition and variation (except
reprintings) of a book. [source: wikipedia]";
reference
"https://en.wikipedia.org/wiki/International_Standard_Book_Number";
}
container authors {
list author {
key name;
leaf name {
type string;
}
leaf account-id {
type uint32 {
range 1001..max;
}
}
}
}
container books {
list book {
key title;
leaf title {
type string;
}
leaf author {
type leafref {
path /authors/author/name;
}
}
leaf language {
type language-type;
}
list format {
key isbn;
leaf isbn {
type ISBN-10-or-13;
}
leaf format-id {
mandatory true;
type identityref {
base format-idty;
}
}
leaf price {
type decimal64 {
fraction-digits 2;
}
units sim-dollar;
}
}
}
}
container users {
list user {
key user-id;
leaf user-id {
type string;
}
leaf name {
type string;
}
}
}
}
Describing Possible Events
Beyond configuration changes that flow from the client to the server, events may occur on either side, and the other party may need to react to them. Information about such events also needs to be part of the YANG contract. When a client informs a server, it’s called an action or rpc. Events taking place on a server are sent to the client as a notification.
Actions and RPCs
The model you have may serve you well as a book catalog and to store information about users (buyers) and authors, but it still lacks something quite fundamental. There is still no way for a client application to request a purchase to take place. For this, you need a YANG action or rpc. RPC stands for remote procedure call, a computer science term for a client asking a server to perform a specified operation.
In YANG, the only difference between an rpc and an action is that an rpc can only be declared at the top level of the YANG model (that is, outside all containers, lists, and so on). An action, on the other hand, cannot be used at the top level, but only inside containers, lists, and so on. Otherwise, they mean exactly the same thing. There are really only historic reasons for separating the two. The action keyword is new in YANG 1.1 and does not exist in YANG 1.0, so many YANG models in the field use only rpc.
When writing new YANG models, use action when the operation you are defining acts on a specific node (object) in the YANG tree and use rpc if the operation cannot be associated with some specific node.
When defining an action (or rpc) in a YANG model, you can specify an input and output parameter section inside the YANG action.
If you elected to make an rpc, which must live outside all containers and lists, the input section for your purchase action might contain a leafref to the buying user, a leafref to which title and format you want to buy, and the number-of-copies to order.
Invoke rpc /purchase with the following input:
user: “janl”
title: “The Neverending Story”
format: “bz:paperback”
number-of-copies: 1
If instead you elect to make an action, you could define the action inside the user list. This way, the user that purchases the book is specified in the path to the action. Perhaps this could be called a bit more object oriented?
Invoke action /users/user[name=“janl”]/purchase:
title: “The Neverending Story”
format: “bz:paperback”
number-of-copies: 1
Notice that there is no major difference between the two forms. You can always turn an action into an rpc by adding a leafref to the object the action operates on. The reverse is not always true, since there are rpcs that don’t operate on any object in the YANG model. A prime example is adding an rpc reboot. There may be no natural place for that rpc within the YANG structure. In this case, it would not fit naturally within /users, certainly not /books, and not /authors.
The action output parameter section lists the data that will come out of the action.
Example 3-13 shows what a complete purchase action might look like. As defined here, this action definition lives inside list user. It could equally have been an rpc (add leafref to user) or defined inside book (remove the leafref to title and format and add a leafref to user). Object orientation pundits may still prefer this model layout since the user can be thought of as the “active part” that invokes the action.
Example 3-13 Action Purchase with Input and Output Data
action purchase {
input {
leaf title {
type leafref {
path /books/book/title;
}
}
leaf format {
type leafref {
path /books/book/format/format-id;
// Reviewer note: This should be improved
}
}
leaf number-of-copies {
type uint32 {
range 1..999;
}
}
}
output {
choice outcome {
case success {
leaf success {
type empty;
description
"Order received and will be sent to specified user.
File orders are downloadable at the URL given below.";
}
leaf delivery-url {
type string;
description
"Download URL for file deliveries.";
}
}
leaf out-of-stock {
type empty;
description
"Order received, but cannot be delivered at this time.
A notification will be sent when the item ships.";
}
leaf failure {
type string;
description
"Order cancelled, for reason stated here.";
}
}
}
}
In this case, the output is modeled as a choice. A YANG choice lists a number of alternatives, of which (at most) one can be present. Each of the alternatives can be listed as a case inside the choice. The case keyword can be skipped if the case contains only a single YANG element, like a single leaf, container, or list.
With the choice statement in Example 3-13, there are three different cases that the action can return:
leafsuccess and/or delivery-url.leafout-of-stockleaffailure
There is actually a fourth case as well. With this YANG model, it would also be valid to return nothing. The choice was not marked mandatory true, so it is not necessarily present at all.
The choice and case nodes are YANG schema nodes. This means that they are invisible in the data tree. When the action returns, it simply returns the contents of one of the case statements, and never mentions anything about choice outcome or case success. When specifying paths to nodes (for example, in a leafref), always omit schema nodes.
Another first encounter in Example 3-13 is the type empty. In YANG, leafs of type empty have no value (hence the name), but they can exist or not exist. A type empty leaf can thus be created and deleted, but not assigned. Type empty leafs are often used to indicate flag conditions, such as success or enabled. The delivery-url and failure are modeled as strings, as opposed to empty, because they return the URL and failure reason, respectively.
Notifications
Notice in the purchase action the possibility that the store is out of stock at the moment. In this scenario, the order is still recorded and will be delivered at a later time. In this case, a notification is sent out to the buyer when the delivery is about to actually happen. This notification shipping includes a reference to the buying user, the title and delivery format of the book, as well as the number of copies.
Notifications are always defined at the top level, outside any containers, lists, and so on, like rpcs, as they’re independent of any YANG objects. In this case, most of the data is pointers into the configuration data model, as seen in Example 3-14.
Example 3-14 Notification Shipping with Output Data
notification shipping {
leaf user {
type leafref {
path /users/user/name;
}
}
leaf title {
type leafref {
path /books/book/title;
}
}
leaf format {
type leafref {
path /books/book/format/format-id;
// Reviewer note: This should be improved
}
}
leaf number-of-copies {
type uint32;
}
}
The way the leafref in leaf format is constructed is not perfect (more on that in the next section).
Before you publish the improved model, add a revision statement, as shown in Example 3-15.
Example 3-15 YANG Module Revision Statement 2018-01-03
revision 2018-01-03 {
description
"Added action purchase and notification shipping.";
}
Separating Configuration from Operational Data
The management at BookZone is quite happy with the progress on standardizing the book catalog. One thing they still miss, however, is some business intelligence and statistics to improve the relevance of titles to book buyers. In particular, a popularity metric would be nice to have for each title.
The popularity metric is not something that a manager could configure; instead, this is operational data—data generated by the system as a reflection of events taking place. This is typically information that a manager would want to monitor or use in troubleshooting activities. In YANG, such data is marked as config false to highlight the operational nature, as opposed to configuration data, which is flagged with config true.
Leafs, containers, lists, and so on that are marked config false can be placed anywhere in a YANG model—at the top level or deep inside some structure. Once config false is applied to a YANG container, list, and so on, everything inside that structure is config false. It is not possible to add a config true leaf somewhere in a config false list, for example. This is quite natural if you think about it. It wouldn’t work to have configuration come and go based on some operational state in the system.
Example 3-16 illustrates leaf popularity added in /books/book. It is marked config false to indicate that it is a status value, not something a manager can set/configure/order.
Example 3-16 Operational Leaf Popularity Added Inside List book
container books {
list book {
...
leaf popularity {
config false;
type uint32;
units copies-sold/year;
description
"Number of copies sold in the last 12-month period";
}
Naturally, the system that implements this YANG model needs to have some sort of code that provides the current value whenever a client asks to see it. Alternatively, the system must tie this to a database that delivers the value on request, but needs to be updated whenever things change.
Another metric the BookZone management is asking for is inventory data per book: the total number of items in stock, the number reserved by a potential buyer, and the number immediately available for delivery.
Example 3-17 provides a model for this.
Example 3-17 Operational Container number-of-copies Added Inside List format
container books {
list book {
...
list format {
...
container number-of-copies {
config false;
leaf in-stock {
type uint32;
}
leaf reserved {
type uint32;
}
leaf available {
type uint32;
}
}
If a container, list, leaf or other YANG element is not marked with either config trueconfig false, it inherits its “config-ness” from its parent. In Example 3-17, leafs in-stock, reserved, and available are config false because the container number-of-copies is config false. The top-level module statement of every YANG module is considered config true, so unless there are any config false statements in a YANG module, everything in it is config true.
The next feature to incorporate in your YANG module is a purchase history for each user. A natural way to organize this is to place a list of purchased items under the /users/user list. Each purchase list item would have a reference to the title and format of the purchased item, the number of copies purchased, and a date.
Example 3-18 is a suggested list purchase-history placed inside the existing list user.
Example 3-18 Operational List purchase-history Added Inside List user
container users {
list user {
...
list purchase-history {
config false;
key "title format";
uses title-format-ref;
leaf transaction-date-time {
type yang:date-and-time;
}
leaf copies {
type uint32;
}
}
}
}
This list happens to have two keys: the title and format. Both are needed to specify a particular row in this list. When you have YANG lists with multiple keys, you specify the key names by separating them by a space inside quotes in the YANG key statement.
For config false data, YANG allows keyless lists, which are simply lists without any key. Since a keyless list has no key, there is no way to navigate in this data or ask for parts of it. From a client perspective, the list needs to be read in its entirety or not at all. There is no way to read just the first 20 entries or to continue to the second page if the operator hits the Page Down key. This can lead to hugely inefficient management protocol exchanges. Do not use keyless lists if there is any way to avoid it.
The uses statement is another keyword you need to be acquainted with. Quite often when YANG modeling, you will notice some collection of leafs or a deeper structure with multiple lists that appears in more than one place in the model. In this case, it may be convenient to define this reusable collection in a YANG grouping. You can then recall this grouping wherever needed without repeating everything inside multiple times. The uses keyword is used to instantiate a copy of such a grouping. You can think of a grouping as a kind of macro. Actually, groupings are more than just macros, but we won’t digress into those details.
Following the uses keyword is the name of the grouping to recall (title-format-ref in this case). Obviously, you need to point to a title and book format-id in the purchase history. As you might recall, you already wrote some YANG for this in the shipping notification. Cut the YANG lines from there and replace them with the uses statement shown in Example 3-19.
Example 3-19 The Notification shipping with Output Data
notification shipping {
leaf user {
type leafref {
path /users/user/name;
}
}
uses title-format-ref;
leaf number-of-copies {
type uint32;
}
}
Then paste the cut lines into a grouping, as shown in Example 3-20.
Example 3-20 Grouping title-format-ref with Commonly Occurring Reference Leafs
grouping title-format-ref {
leaf title {
type leafref {
path /books/book/title;
}
}
leaf format {
type leafref {
path /books/book/format/format-id;
// Reviewer note: This should be improved
}
}
}
One of the merits of using groupings like this is that the number of lines in the YANG module gets smaller, so you have less model text to maintain. Another benefit is that if you improve the grouping, the improvement immediately applies everywhere. Just make sure that is what you want! A downside with using too many groupings is that it sometimes gets hard to follow multiple levels of uses statements crisscrossing the model. In other words, do not go overboard with groupings.
Actually, the same references to title and format are also found in the purchase action. Update it to use your new grouping to be consistent (and save a few lines), as shown in Example 3-21.
Example 3-21 Action purchase That Uses the Grouping title-format-ref
container users {
list user {
key user-id;
...
action purchase {
input {
uses title-format-ref;
leaf number-of-copies {
type uint32 {
range 1..999;
}
}
}
By now, you are using the grouping in three places: in the config false list purchase-history, the notification shipping, and the action purchase. This is reuse in practice! Example 3-22 illustrates the latest tree diagram of your model. In tree diagrams, all groupings are expanded so that you can see the full tree.
Example 3-22 YANG Tree Representation of the Module with Revision 2018-01-04
module: bookzone-example
+--rw authors
| +--rw author* [name]
| +--rw name string
| +--rw account-id? uint32
+--rw books
| +--rw book* [title]
| +--rw title string
| +--rw author? -> /authors/author/name
| +--rw language? language-type
| +--ro popularity? uint32
| +--rw format* [isbn]
| +--rw isbn ISBN-10-or-13
| +--rw format-id identityref
| +--rw price? decimal64
| +--ro number-of-copies
| +--ro in-stock? uint32
| +--ro reserved? uint32
| +--ro available? uint32
+--rw users
+--rw user* [user-id]
+--rw user-id string
+--rw name? string
+---x purchase
| +---w input
| | +---w title? -> /books/book/title
| | +---w format? -> /books/book/format/format-id
| | +---w number-of-copies? uint32
| +--ro output
| +--ro (outcome)?
| +--:(success)
| | +--ro success? empty
| | +--ro delivery-url? string
| +--:(out-of-stock)
| | +--ro out-of-stock? empty
| +--:(failure)
| +--ro failure? string
+--ro purchase-history* [title format]
+--ro title -> /books/book/title
+--ro format -> /books/book/format/format-id
+--ro transaction-date-time? yang:date-and-time
+--ro copies? uint32
notifications:
+---n shipping
+--ro user? -> /users/user/name
+--ro title? -> /books/book/title
+--ro format? -> /books/book/format/format-id
+--ro number-of-copies? uint32
Before you publish the new version, add a revision statement, as shown in Example 3-23.
Example 3-23 Revision Statement 2018-01-04
revision 2018-01-04 {
description
"Added status information about books and purchases, see
/books/book/popularity
/books/book/formats/number-of-copies
/users/user/purchase-history
Turned reference to book title & format
into a grouping, updated references in
/users/user/purchase
/shipping";
}
Constraints Keep Things Meaningful
More than anything else, it is essential that a contract is clear and precise. Both sides should know what to expect, and few surprises should be discovered down the line. This builds confidence and utility. YANG is a contract, so for clear interoperability, it is essential that both sides understand the terms and that the contract is precise in its details.
A common beginner’s mistake is to bring a YANG model where all leafs are declared as strings or integers with no constraints or precision. That model is seldom useful, since there are pretty much always a lot of formats, ranges, and dependencies that must be respected. How would a client know what to do, and what is the chance that two server implementers would use the same way of encoding the information in this case?
Sometimes really complex business logic is required to determine what is valid and what is not. Just as you do not want to encode your entire business process within a contract, there are plenty of details not to model in YANG. Find a balance. A really long and super-detailed contract may be useful at times, whereas a more open and sloppier one is easier to write. Generally, the highest global utility is when you find a balance where the contract is fairly precise, but not too long or dense.
You have already used mandatory true, which is one kind of constraint on valid data. Since YANG model leafs are optional by default, another common beginner’s mistake is to not describe what it means if the leaf is not present. Marking the leaf mandatory true is one way out. Another one is to add a default statement to the leaf; then it is clear how to interpret the leaf if nobody has touched it. At the very least, the modeler ought to specify in the leaf description the system behavior when the leaf has no value.
Coming back to the bookzone-example, in the purchase action, you need to know the number of book copies to deliver. If not specified, assume the buyer means one copy. In order to make this clear, add a default 1 statement to leaf number-of-copies, as shown in Example 3-24.
Example 3-24 Action purchase with default 1 for Leaf number-of-copies
action purchase {
input {
uses title-format-ref;
leaf number-of-copies {
type uint32 {
range 1..999;
}
default 1;
}
In your book catalog, the price leaf is optional. You could go ahead and make it mandatory so that whenever a book is being added, a price must be specified. This is not how BookZone likes to work, though. BookZone wants to sometimes add entries before the item in question is orderable and details like price were worked out. Therefore, keep price optional, just make sure the item isn’t orderable until it has a price. This is easy to do by adding a must statement. A must statement in YANG prescribes a constraint that must hold true at all times. In this case, since you are placing it inside an action, the constraint must hold true at the time the action is invoked.
The purchase action contains a reference to the title and format through the uses title-format-ref construct, which brings in the grouping title-format-ref. The format leafref points to the /books/book/format/format-id of the book you are looking at. If you follow this pointer, you can navigate to the price of the book in this particular book format by backing up a level and going into the price leaf. Then you can check that it is greater than zero.
A common beginner’s mistake is to make a must statement that just points directly to the leaf that is to be tested (in this case, price), as shown in this snippet:
must "/books/book/format/price>0"
Even if this expression seems simple enough, its simplicity is deceiving. What this means is that the purchase action can only be invoked if there is any book (at least one) that has a price greater than zero. Here, you only really care if the particular book you are ordering has a price, so you must look up the price of this particular book. Therefore, the must statement you want to add is shown in Example 3-25.
Example 3-25 A must Statement with a Proper deref()
action purchase {
input {
must "deref(format)/../price>0" {
error-message "Item not orderable at this time";
}
uses title-format-ref;
leaf number-of-copies {
type uint32 {
range 1..999;
}
default 1;
}
Let’s look at what the must statement really means, step by step. First, the value that follows the keyword must is an expression in the XPath language. XPath is a language that comes from the family of languages around the XML standard. In YANG, XPath is used to point at other parts of the model, and here it is used as a true/false Boolean expression for something that needs to be true for the action to be valid and executable.
The expression needs to be within quotes because it contains special characters. The first part is an XPath function call to an XPath function called deref(). This function follows a leafref pointer, which is specified as input to the function within the parentheses (in this case, leaf format). Leaf format is not immediately visible in the preceding YANG because it’s inside the grouping called with uses title-format-ref. Calling deref(format) takes you to where format points: a particular book of a particular format, pointing to its format-id. The full path is /books/book/format/format-id in XPath speak. This appears at #1 in Example 3-26.
Example 3-26 YANG Tree Diagram with XPath Navigation from format-id to price
module: bookzone-example
...
+--rw books
| +--rw book* [title]
| +--rw title string
| +--rw author? -> /authors/author/name
| +--rw language? language-type
| +--ro popularity? uint32
| +--rw format* [isbn]
| #2
| +--rw isbn ISBN-10-or-13
| +--rw format-id #1 identityref
| +--rw price? #3 decimal64
From there you want to go up one level into the leaf price. “Go up one level” is expressed with two dots (..), taking your to the position marked #2. Then you want to go down into price with the slash price at the end, shown as position #3. Look at the tree diagram of the /books/book part of the model to see how the expression takes you closer and closer to your target node.
Once you finally reach the price leaf, compare it to see if it is greater than zero. If price does not exist or if it is not greater than zero, the must statement is not true, and all parties can agree that the purchase action cannot be executed according to the contract (a.k.a. the YANG model). This is what clarity and precision in the YANG model means.
Another detail to consider is the purchase action’s reference to the format-id. The format-id is the name of the format of the book, such as paperback, hardcover, or mp3. This is not the key to list format, so in principle, there could exist several ISBNs for a given book title with the same format. This could happen, for example, if a given title is published by more than one publisher. ISBN 9780140386332 and 9780140317930 are both paperback editions in English of The Neverending Story, just to give one example.
You could have (some might argue should have) made the purchase action point out an ISBN instead of a title and format, but you remember that someone in the BookZone marketing department made the following comment: “People buy books by title and format, and care little about ISBNs. As much as those numbers are loved internally for their clarity, users order by title and format.” This is indeed how the purchase action was modeled earlier in this chapter.
As a consequence, BookZone customers do not have any reliable way of knowing what ISBN they receive if they order The Neverending Story in case there are several ISBNs for the same format (such as for paperback).
Since this is not ideal, BookZone decided that in their book catalog, this should never be allowed to happen. Therefore, BookZone only carries one ISBN for each title and format combination (in this case, only one paperback variant of The Neverending Story).
To enforce this, add a unique statement to the YANG module, as shown in Example 3-27.
Example 3-27 Uniqueness Constraint in a YANG List
container books {
list book {
...
list format {
key isbn;
unique format-id;
leaf isbn {
type ISBN-10-or-13;
}
leaf format-id {
mandatory true;
type identityref {
base format-idty;
}
}
This enforces that within any given list book entry, no two list entries in list format have the same value for leaf format-id. New list format entries that violate this rule can be added within a transaction, but when the transaction is committed, each entry must have a unique format-id value.
Mandatory and Default Data
Another similar case is leaf author under book. No book should ever be listed without an author, so this leaf should be mandatory. Therefore, you need to add that in. Beyond that, BookZone does not allow books to be orderable unless the author of the book has an account set up with the finance department.
You could of course make leaf account-id under list author be mandatory as well so that no author can ever be added until the account setup is completed. The BookZone finance department does not like that idea, as it does not match their internal process. Instead, you must ensure that whenever a book is added to list book, at that point the book’s author must have an account-id with finance. This is done with another must statement, as shown in Example 3-28.
Example 3-28 Leaf author with a must Statement Checking If the Author’s account-id Is Set
container books {
list book {
...
leaf author {
type leafref {
path /authors/author/name;
}
must 'deref(current())/../account-id' {
error-message
"This author does not yet have an account-id defined.
Books cannot be added for authors lacking one.
The Finance Department is responsible for assigning
account-id's to authors.";
}
mandatory true;
}
Let’s decode the must statement in Example 3-28. You already know the deref() XPath function follows a leafref pointer. This time, however, the argument to deref is another XPath function call. The current() XPath function returns the YANG element that the must statement is sitting on (leaf author in this case). The leafref points to /authors/author/name. From there, you need to move up one level to get into the list author, then onward into account-id.
In XPath, just pointing to a leaf without comparing it to anything turns into an existence test, so the must expression basically says “the current author must have an account-id.” The error-message statement is simply a message displayed to the operator in case the must statement condition is not fulfilled.
Conditional Content
XPath expressions also come in handy for the YANG when statement. These statements determine when a particular YANG construct is relevant and make it disappear when it is not relevant.
An example of this is the container number-of-copies, which contains three leafs with information about how many books are in stock, how many are reserved by a potential customer, and how many are available for sale. These parameters obviously only apply to the books with a physical form factor. For those books based on file delivery, this container is simply not relevant.
What you want here is for this container to be disregarded when the book format is derived from the bz:file-idty identity, as shown in Example 3-29. Here is where the YANG identities get to shine, because using them you can now make a rule that applies to a whole range of different book formats. This range can even grow in the future while the rule still works.
Example 3-29 Container That Is Not Relevant for Any File-Based Delivery Formats
container books {
list book {
...
list format {
...
container number-of-copies {
when 'not(derived-from(../format-id, "bz:file-idty"))';
config false;
leaf in-stock {
type uint32;
}
leaf reserved {
type uint32;
}
leaf available {
type uint32;
}
}
In the when expression, notice two new XPath functions being used. The first is not(), which does exactly what it sounds like: It negates the logical value of the argument. The second is derived-from(), which checks whether some leaf of type identityref has a value that is derived from the given identity. In this case, the ../format-id leaf (which is an identityref) can have the value bz:mp3, bz:pdf, or bz:epub—or any other identity value added in the future and based directly on bz:file-idty or indirectly on any other identity derived from it in turn.
Properly Following Pointers
As you may have noticed, there was a “reviewer note” that the format leafref should be improved somehow. Before revealing the fix, let’s have a look at the problem.
If you look at leaf title, it can point to the title of any book. That’s exactly what you want: The user should be able to purchase any title. So far so good. Then you have leaf format. With a simple leafref statement, it can point to any book format. This may sound right at first, but this is actually too liberal—you could end up with nonexistent combinations of the titles and formats.
Valid values for format should only be formats available for the particular book pointed to by title. Table 3-3 includes sample data, reflecting the inventory of a particular BookZone store.
Table 3-3 List Format Inside list book in a Tabular Representation
title |
format |
|---|---|
The Hitchhiker’s Guide to the Galaxy |
bz:paperback |
bz:hardcover |
|
The Neverending Story |
bz:mp3 |
Now let’s consider leaf title. It’s a leafref to /books/book/title, which means any value that exists in that list column is a valid value for this leaf. With this sample data, that could be either “The Hitchhiker’s Guide to the Galaxy” or “The Neverending Story”.
If the leaf format is modeled as a simple leafref pointing to /books/book/formats/format-id, any book format value that exists in the format-id column is allowed. With this sample data, that is “bz:paperback”, “bz:hardcover”, or “bz:mp3”.
The problem with this simple YANG model is that it allows title to be “The Hitchhiker’s Guide to the Galaxy” while format is “bz:mp3”. Or title is “The Neverending Story” and format is “bz:paperback” or “bz:hardcover”. These are combinations that are not available in the BookZone store inventory!
The YANG model is too lax here, allowing combinations of values that are not meaningful. YANG is not expressive enough to describe all possible business logic constraints on data that you could ever encounter. You may need to implement business logic beyond what you can describe in YANG. When you use constraints described in YANG, however, it brings clarity to the contract between the client and server. Common requirements like in the case at hand can certainly be properly described in YANG.
Clearly, the user should be able to select any title in list book, so title being a leafref to any book is fine. The problem with the current model is that leaf format can point to any format-id under list book/format, not just the formats actually available for the current title.
The way to fix this is to change the leafref in leaf format so that it dereferences title. Leaf title points to a specific book, and if the format leafref followed that pointer to the specific book, and only allowed selecting a format-id present under that specific book, you would be home free.
By using an XPath predicate, i.e. an XPath filter inside square brackets, the format leafref can follow leaf title to the book in question and limit the valid values to an existing format-id under that book. Examples 3-25 and 3-28 showed XPath expressions with similar purpose using the XPath deref() function. Unfortunately the YANG specification does not formally allow the use of deref() in leafref paths, so the traditional XPath predicate approach must be used here, arriving at the exact same functionality in a more verbose way. There is no reason why the YANG specification doesn’t allow deref() in leafref paths, but the fact is that it doesn’t. Still, many tools do allow it.
Because dereferencing and XPath navigation are not easy topics to grasp, let’s review this in detail. Example 3-30 shows what the grouping that provides a pointer to a particular title and format-id combination becomes.
Example 3-30 XPath Navigation from format to title
grouping title-format-ref {
#2
leaf title { #3
type leafref {
path /books/book/title;
}
}
leaf format { #1
type leafref {
path /books/book[title=current()/../title]/format/format-id;
// The path above expressed using deref():
// path deref(../title)/../format/format-id;
}
}
}
The point with leaf format is to allow the operator to pick one format-id value from the list of available values in /books/book/format/format-id. Just pointing to that path gives the operator the possibility to choose from all valid values for any book, so in order to limit the choices to the formats available for the current book, a filter expression is required. This way the leafref can only point to format-id values relevant for the current title.
The leafref path in leaf format is decoded as follows. First, look at all /books/book entries. The expression in the square brackets is the predicate (filter), and selects only books that have a title leaf that equals the value to the right of the equals sign. That will be a single book. The value to the right of the equals sign navigates from the current node (#1: leaf format), up one level (#2: wherever this grouping is used), and then into leaf title (#3). There might be many such title nodes in the data tree, since /books/book/format is a list. Essentially, this expression allows any format-id value from the list format within the current /books/book.
In a comment, Example 3-30 also shows the exact same path and filter function expressed using deref(): The first piece is deref(), an XPath function that follows a leafref. In this case, the leafref to follow is the one in leaf title. A path pointing out leaf title is provided as an argument (that is, inside the parentheses) to the deref() function. The two dots mean “go up one level” from #1 to #2, just like in a directory tree path. This moves you out of leaf format. The slash and word title moves us into leaf title, at #3. The deref(../title) part of the expression takes you to wherever title points. This will be a particular title in list /books/book (see #4 in Example 3-31).
Example 3-31 YANG Tree Instance Data Diagram with XPath Navigation from title to format-id
Instance data diagram:
+--rw books
| +--rw book [0]
| | +--rw title "The Hitchhiker's Guide to the Galaxy"
| | +--rw author? "Douglas Adams"
| | +--rw language? english
| | +--rw format [0]
| | | +--rw isbn "0330258648"
| | | +--rw format-id bz:paperback
| | | +--rw price? 22.00
| | +--rw format [1]
| | +--rw isbn "9781400052929"
| | +--rw format-id bz:hardcover
| | +--rw price? 31.50
| +--rw book [1]
| #5
| +--rw title #4 "The Neverending Story"
| +--rw author? "Michael Ende"
| +--rw language? english
| +--rw format [0]
| #6
| +--rw isbn "9781452656304"
| +--rw format-id #7 bz:mp3
| +--rw price? 29.95
Use the instance data diagram in Example 3-31 to follow the navigation in Example 3-30. An instance data diagram is a cross between the YANG tree diagram shown earlier and a database dump. Here you see some sample database data in the YANG tree structure.
If, for example, the user specified “The Neverending Story” for title, the leafref expression so far would point to /books/book[title=“The Neverending Story”]/title (see #4).
However, you want it to point to /books/book[title=“The Neverending Story”]/format/format-id, so you go up one level (#5), then into list format (#6), and finally into leaf format-id (#7).
Now the YANG model is safe and sound. Since you updated the grouping, the fix immediately gets to all three places where you reference a title and format. This is the beauty of the grouping concept.
It is a very common YANG modeling mistake to think that a relative path and an absolute path pointing to the same leaf are equivalent. As demonstrated in Example 3-31, in YANG, they often are not. A relative path such as deref(../title)/../format/format-id does not evaluate to the same set of values as /books/book/title/format/format-id, even though they point to the same format-id leaf, unless some XPath predicates (filters) are added.
Schema Nodes Don’t Count
Another one of the most common errors when constructing or walking XPath expressions is to mistakenly pay attention to schema nodes that are not data nodes. When you’re constructing the path in an XPath expression, only data nodes should be counted and referenced. Basically, there are a bunch of keywords in a YANG schema that must be disregarded in paths since they work on a “meta level.” Think of them as macros, if that helps. These include input, output, choice, case, grouping, and uses.
To illustrate this point, add some payment methods to /users/user and a way to specify which payment method to use in the purchase action, as shown in Example 3-32.
Example 3-32 Leafref path Crossing a Schema Node
container users {
list user {
key user-id;
leaf user-id {
type string;
}
leaf name {
type string;
}
container payment-methods {
list payment-method {
key "method id";
leaf method {
type enumeration {
enum bookzone-account;
enum credit-card;
enum paypal;
enum klarna;
}
}
leaf id {
type string;
}
}
}
action purchase {
input {
must "deref(format)/../price>0" {
error-message "Item not orderable at this time";
}
uses title-format-ref;
leaf number-of-copies {
type uint32 {
range 1..999;
}
default 1;
}
container payment {
leaf method {
type leafref {
path ../../../payment-methods/payment-method/method;
}
}
leaf id {
type leafref {
path "../../../payment-methods/"+
"payment-method[method=current()/../method]/id";
// The path above expressed using deref():
// path deref(../method)/../id;
}
}
}
}
How to construct that leafref path in leaf method? The first .. is used to get out of leaf method. The second .. is used to get out of container payment. Then you have the keyword input, which does not count for the path, as it is not a data node. The third .. is thus used to get out of action purchase. From there, you dive into container purchase-methods, into list purchase-method, and finally into leaf method.
In order to verify that the XPath expressions are correct, it is strongly advised that you use proper tools. Sadly, many YANG tools (compilers) historically have never checked the XPath expressions for correctness. Some still don’t. This has resulted in many YANG modelers believing that their modules were good since the compiler was happy, when in fact they still contained a good deal of broken XPath.
The final, complete model is shown in Example 3-33.
Example 3-33 Complete bookzone-example YANG Module with Revision 2018-01-05
module bookzone-example {
yang-version 1.1;
namespace 'http://example.com/ns/bookzone';
prefix bz;
import ietf-yang-types {
prefix yang;
}
organization
"BookZone, a fictive book store chain";
contact
"YANG book project: https://github.com/janlindblad/bookzone
Editor: Jan Lindblad
<mailto:janl@tail-f.com>";
description
"BookZone defines this model to provide a standard interface for
inventory browser and management applications.
Copyright (c) 2018 the YANG book project and the persons
identified as authors of the code. All rights reserved.
Redistribution and use in source and binary forms, with or
without modification, is permitted pursuant to, and subject
to the license terms contained in, the Simplified BSD License
set forth in Section 4.c of the IETF Trust's Legal Provisions
Relating to IETF Documents
(http://trustee.ietf.org/license-info).";
revision 2018-01-05 {
description
"Added constraints that
- author needs to have an account set before listing a book
- number of copies in stock only shows for physical items
- makes a book not orderable unless it has a price
- book leafrefs are chained correctly
Added /users/user/payment-methods and a way to choose which
one to use in action purchase.";
}
revision 2018-01-04 {
description
"Added status information about books and purchases, see
/books/book/popularity
/books/book/formats/number-of-copies
/users/user/purchase-history
Turned reference to book title & format
into a grouping, updated references in
/users/user/purchase
/shipping";
}
revision 2018-01-03 {
description
"Added action purchase and notification shipping.";
}
revision 2018-01-02 {
description
"Added book formats, authors and users, see
/books/book/format
/authors
/users";
}
revision 2018-01-01 {
description "Initial revision. A catalog of books.";
}
typedef language-type {
type enumeration {
enum arabic;
enum chinese;
enum english;
enum french;
enum moroccan-arabic;
enum swahili;
enum swedish;
// List not exhaustive in order to save space
}
description
"Primary language the book consumer needs to master "+
"in order to appreciate the book's content";
}
identity format-idty {
description "Root identity for all book formats";
}
identity paper {
base format-idty;
description "Physical book printed on paper";
}
identity audio-cd {
base format-idty;
description "Audiobook delivered as Compact Disc";
}
identity file-idty {
base format-idty;
description "Book delivered as a file";
}
identity paperback {
base paper;
description "Physical book with soft covers";
}
identity hardcover {
base paper;
description "Physical book with hard covers";
}
identity mp3 {
base file-idty;
description "Audiobook delivered as MP3 file";
}
identity pdf {
base file-idty;
description "Digital book delivered as PDF file";
}
identity epub {
base file-idty;
description "Digital book delivered as EPUB file";
}
typedef ISBN-10-or-13 {
type union {
type string {
length 10;
pattern '[0-9]{9}[0-9X]';
}
type string {
length 13;
pattern '97[89][0-9]{10}';
}
}
description
"The International Standard Book Number (ISBN) is a unique
numeric commercial book identifier.
An ISBN is assigned to each edition and variation (except
reprintings) of a book. [source: wikipedia]";
reference
"https://en.wikipedia.org/wiki/International_Standard_Book_Number";
}
grouping title-format-ref {
leaf title {
type leafref {
path /books/book/title;
}
}
leaf format {
type leafref {
path /books/book[title=current()/../title]/format/format-id;
// The path above expressed using deref():
// path deref(../title)/../format/format-id;
}
}
}
container authors {
list author {
key name;
leaf name {
type string;
}
leaf account-id {
type uint32 {
range 1001..max;
}
}
}
}
container books {
list book {
key title;
leaf title {
type string;
}
leaf author {
type leafref {
path /authors/author/name;
}
must 'deref(current())/../account-id' {
error-message
"This author does not yet have an account-id defined.
Books cannot be added for authors lacking one.
The Finance Department is responsible for assigning
account-id's to authors.";
}
mandatory true;
}
leaf language {
type language-type;
}
leaf popularity {
config false;
type uint32;
units copies-sold/year;
description
"Number of copies sold in the last 12-month period";
}
list format {
key isbn;
unique format-id;
leaf isbn {
type ISBN-10-or-13;
}
leaf format-id {
mandatory true;
type identityref {
base format-idty;
}
}
leaf price {
type decimal64 {
fraction-digits 2;
}
units sim-dollar;
}
container number-of-copies {
when 'not(derived-from(../format-id, "bz:file-idty"))';
config false;
leaf in-stock {
type uint32;
}
leaf reserved {
type uint32;
}
leaf available {
type uint32;
}
}
}
}
}
container users {
list user {
key user-id;
leaf user-id {
type string;
}
leaf name {
type string;
}
container payment-methods {
list payment-method {
key "method id";
leaf method {
type enumeration {
enum bookzone-account;
enum credit-card;
enum paypal;
enum klarna;
}
}
leaf id {
type string;
}
}
}
action purchase {
input {
must "deref(format)/../price>0" {
error-message "Item not orderable at this time";
}
uses title-format-ref;
leaf number-of-copies {
type uint32 {
range 1..999;
}
default 1;
}
container payment {
leaf method {
type leafref {
path ../../../payment-methods/payment-method/method;
}
}
leaf id {
type leafref {
path "../../../payment-methods/"+
"payment-method[method=current()/../method]/id";
// The path above expressed using deref():
// path deref(../method)/../id;
}
}
}
}
output {
choice outcome {
case success {
leaf success {
type empty;
description
"Order received and will be sent to specified user.
File orders are downloadable at the URL given below.";
}
leaf delivery-url {
type string;
description
"Download URL for file deliveries.";
}
}
leaf out-of-stock {
type empty;
description
"Order received, but cannot be delivered at this time.
A notification will be sent when the item ships.";
}
leaf failure {
type string;
description
"Order cancelled, for reason stated here.";
}
}
}
}
list purchase-history {
config false;
key "title format";
uses title-format-ref;
leaf transaction-date-time {
type yang:date-and-time;
}
leaf copies {
type uint32;
}
}
}
}
notification shipping {
leaf user {
type leafref {
path /users/user/name;
}
}
uses title-format-ref;
leaf number-of-copies {
type uint32;
}
}
}
Example 3-34 shows a tree representation of the model.
Example 3-34 Complete bookzone-example YANG Tree with Revision 2018-01-05
module: bookzone-example
+--rw authors
| +--rw author* [name]
| +--rw name string
| +--rw account-id? uint32
+--rw books
| +--rw book* [title]
| +--rw title string
| +--rw author -> /authors/author/name
| +--rw language? language-type
| +--ro popularity? uint32
| +--rw format* [isbn]
| +--rw isbn ISBN-10-or-13
| +--rw format-id identityref
| +--rw price? decimal64
| +--ro number-of-copies
| +--ro in-stock? uint32
| +--ro reserved? uint32
| +--ro available? uint32
+--rw users
+--rw user* [user-id]
+--rw user-id string
+--rw name? string
+--rw payment-methods
| +--rw payment-method* [method id]
| +--rw method enumeration
| +--rw id string
+---x purchase
| +---w input
| | +---w title? -> /books/book/title
| | +---w format? -> /books/book[title=current()/../title]/
format/format-id
| | +---w number-of-copies? uint32
| | +---w payment
| | +---w method? -> ../../../payment-methods/payment-method/method
| | +---w id? -> ../../../payment-methods/payment-method[method=
current()/../method]/id
| +--ro output
| +--ro (outcome)?
| +--:(success)
| | +--ro success? empty
| | +--ro delivery-url? string
| +--:(out-of-stock)
| | +--ro out-of-stock? empty
| +--:(failure)
| +--ro failure? string
+--ro purchase-history* [title format]
+--ro title -> /books/book/title
+--ro format -> /books/book[title=current()/../title]/
format/format-id
+--ro transaction-date-time? yang:date-and-time
+--ro copies? uint32
notifications:
+---n shipping
+--ro user? -> /users/user/name
+--ro title? -> /books/book/title
+--ro format? -> /books/book[title=current()/../title]/format/
format-id
+--ro number-of-copies? uint32
Augmenting, Extending, and Possibly Deviating
The company is grateful for all the work you have done in building up the BookZone YANG model. BookZone is now thriving thanks to the well-defined interface to all subsidiary stores.
One day, one of the BookZone employees had an idea for a new business venture. She presented the idea to the management team, and soon enough they decided to spin off a side venture to go after this new market. They formed AudioZone, for selling online subscriptions to a library of audio books.
Obviously, having a catalog like the BookZone catalog system is key for this new line of business. Clearly, some tweaks are necessary, though, such as allowing users to rate and review audio books they have listened to, and tracking users’ friends in order to give the app some social aspects.
You could certainly go and edit the BookZone YANG files to add in what is needed for AudioZone, but as the module evolves on both sides, that would soon become a mess. YANG offers a better option. Using the YANG augment statement, you can graft new content in one YANG module onto the context of another module. Thanks to the extensible nature of XML (which stands for Extensible Markup Language, after all), these augmentations can be mapped to XML-based protocols like NETCONF in way that’s really easy to deal with, even for managers who have no prior knowledge about the augmented content.
The first thing you need to do is to create a new AudioZone YANG module. The header is shown in Example 3-35.
Example 3-35 YANG Module audiozone-example
module audiozone-example {
yang-version 1.1;
namespace 'http://example.com/ns/audiozone';
prefix az;
import bookzone-example {
prefix bz;
}
import ietf-yang-types {
prefix yang;
}
organization
"AudioZone, a subsidiary of the fictive book store chain BookZone";
contact
"YANG book project: https://github.com/janlindblad/bookzone
Editor: Jan Lindblad
<mailto:janl@tail-f.com>";
description
"AudioZone defines this model to provide a standard interface for
inventory browser and management applications. This module extends
the bookzone-example.yang module.
Copyright (c) 2018 the YANG book project and the persons
identified as authors of the code. All rights reserved.
Redistribution and use in source and binary forms, with or
without modification, is permitted pursuant to, and subject
to the license terms contained in, the Simplified BSD License
set forth in Section 4.c of the IETF Trust's Legal Provisions
Relating to IETF Documents
(http://trustee.ietf.org/license-info).";
revision 2018-01-09 {
description "Initial revision.";
}
Note how the module uses an import bookzone-example statement to make all the types and definitions in that module available in the current module. The import statement will not copy the contents of that module into this one, unlike #include and such in other languages. It simply makes this module aware of the other module’s existence and makes it legal to refer to content in that module from here.
References to content in the imported module use the prefix given in the import statement (bz in this case), as specified in prefix bz, under the import bookzone-example statement. The prefix given in an import statement is usually the same as the prefix in the header of that module, but that is not a requirement.
Now, you need to add a new book format, streaming, for the content streamed to all clients, as shown in Example 3-36.
Example 3-36 Additional Delivery Format Identity
identity streaming {
base bz:file-idty;
description
"Audiobook streaming.";
}
This declares a new identity in your module. By adding bz: in front of file-idty, it’s clear that you mean an identity found in the bookzone-example module. By simply declaring a new identity and basing it on an identity that directly or indirectly is referenced by some identityrefs in any module, you cause the new value immediately to become a valid option in all those places. Talk about easy to extend!
If you look back at the bookzone-example module, /books/book/format/format-idty is an identityref that allows the leaf format-id to have as a value the name of any identity derived from format-idty. Your new identity, streaming, is based on file-idty, which is based on format-idty. In any system that supports the audiozone-example module, streaming is now a valid value for format-id. In effect, the bookzone-example YANG module is unmodified, yet it has been extended with a new valid value. This happens because the server implements and advertises the audiozone-example YANG module, not because of any YANG import statements. Example 3-37 is a reminder excerpt from bookzone-example showing the format-id identityref leaf.
Example 3-37 Leaf format-id from Example 3-33
leaf format-id {
mandatory true;
type identityref {
base format-idty;
}
}
Now, the next step is to augment the bookzone-example module with some recommendations for each /books/book, as shown in Example 3-38.
Example 3-38 List Recommendations Augmenting List book from Example 3-33
augment /bz:books/bz:book {
list recommendations {
key review-date;
leaf review-date {
type yang:date-and-time;
}
leaf score {
type uint32 {
range 1..5;
}
}
leaf review-comment {
type string;
}
}
}
The augment keyword simply adds the content inside the statement to the location given after the keyword. The target location is usually in a different module, perhaps one created by a standards body. In this case, the target is the bookzone-example module, which you can see since the path uses the bz: prefix.
Augments may also extend the same module they are defined in. This could be useful at times, such as when you want to build up a nice and concise list of something (say, interfaces) and then add details for all the different kinds of interfaces further down in your module.
The final addition needed for the AudioZone venture is a list of friends for each user, as shown in Example 3-39. You can just augment it here by using a simple kind of list called a leaf-list. Leaf-lists are good when you want a list of a single thing. This is a special case for single-column lists that eliminates some typing.
Example 3-39 Leaf-list friends Augmenting List user from Example 3-33
augment /bz:users/bz:user {
leaf-list friends {
type leafref {
path /bz:users/bz:user/bz:name;
}
}
}
With this, you have a complete audiozone-example module that you can load up in your NETCONF server—of course, together with the imported bookzone-example module.
A topic that often comes up is how to think about YANG model modularity. Should all the functionality be defined in one grand module, or is it better with a hundred smaller ones?
You just saw one reason for having separate modules—you are adding to a module developed by a different organization. The most common reason for splitting functionality into different modules is versioning.
If a system with a given YANG interface is upgraded with additional functionality, it is nice if the consumers of the interface can see roughly where it changed and where it is the same. By stepping the version on four of, say, 14 YANG modules provided, you allow the users to quickly determine what areas, if any, they care about being changed (or not). If there are hundreds of modules, understanding this gets harder again.
Extending YANG
The YANG language itself is extensible; you can define your own keywords that describe some aspects of your model that the base YANG language does not capture. For example, people have added code-generation statements, user-documentation-generation directives, additional validation logic, and test-case data to their models. You can read more about the extension keyword in RFC 7950.
Declaring your own keyword is easy. Simply use the YANG extension keyword, followed by the name of your new keyword.
Let’s say BookZone wants to build a function to export some of their YANG lists to an SQL database. You could declare an extension keyword of your own that the BookZone system applications can use to figure out what SQL tables should be used. You could then apply that extension to the relevant lists. This way, the export function would be guided by the YANG modules alone, and not rely on additional files or hardcoding.
In Example 3-40, three new keywords are declared in a new YANG module. Two of them take one argument, and the last one takes none.
Example 3-40 YANG Module Declaring Three extension Keywords
module bookzone-example-extensions {
yang-version 1.1;
namespace 'http://example.com/ns/bookzone-extensions';
prefix bzex;
extension sql-export-to-table {
argument table-name;
}
extension sql-export-to-column {
argument column-name;
}
extension sql-export-as-key;
}
In order to use these new keywords now, all you need to do is import the module in which they are declared (you can skip this if they are declared in the same module as where they are being used) and then refer to the prefixed name of the new keywords, as shown in Example 3-41.
Example 3-41 Import and Usage of extension Keywords from Previous Example
module bookzone-example {
...
import bookzone-example-extensions {
prefix bzex;
}
...
container users {
list user {
bzex:sql-export-to-table users;
key user-id;
leaf user-id {
bzex:sql-export-to-column id;
bzex:sql-export-as-key;
type string;
}
leaf name {
bzex:sql-export-to-column login_name;
type string;
}
container payment-methods {
Many YANG parsers would have no idea what your new keyword means. Since you declared them correctly, well-written YANG parsers will still be able to correctly compile the entire module, ignoring the extension.
In many other languages, similar effects are accomplished by adding comments with particular (odd) formats, or by using pragma statements and such. The downside with this approach is that any misspelled comment or pragma wouldn’t be detected by the compiler, and the whole construct may not be clear to people who don’t recognize the special meaning of the comment.
Note also that it must remain optional for YANG parsers to understand the new keyword and that the module must still be valid while ignoring it. Extensions can therefore not break YANG rules. It is valid, however, to require an understanding of and respect for certain extension keywords as part of implementing a YANG feature. The feature description could very well say that in order to announce this feature, a conforming device must do this or that as indicated by some extensions in the module.
Extension keywords can have zero or one argument to fit in with the general YANG syntax rules. If more information is needed, either multiple extensions could be used or the format of the argument could be something complex, as shown in Example 3-42.
Example 3-42 Two Different Alternatives for Mapping Multiple Pieces of Information Using the YANG extension Keyword
leaf name {
object-relational-map:key "Name";
object-relational-map:type "UnicodeStringType";
type name-type;
}
// -- or like this --
leaf name {
object-relational-map:key-of-type "Name:UnicodeStringType";
type name-type;
}
Deviations
As standards-defining organizations (SDOs) release YANG modules, they have usually thought through the use cases for their modules in good detail. In order to reach consensus, the models tend to include everybody’s favorite use case—and sometimes legacy ways of doing things.
Organizations that implement these standard models may need to release a first version of their code before all the use cases allowed by the YANG model are covered. It may even be decided that certain use cases will never be implemented. The implementation cannot be said to fully implement the model in these cases, but it may still provide great customer value. If an implementation isn’t complete, it’s prudent to make clear what parts aren’t covered using the YANG deviation keyword.
Let’s say an implementor has developed support for ietf-interfaces.yang, but is not (currently) able to support the leaf last-change in the model. Skipping the implementation and then letting people find out by themselves may cause a lot of extra work and irritation for everybody, so that’s not a good option. Writing about it in a release notes document is better, but any tools would still have no clue. A better and more precise option then is to declare this lack of support with a deviation statement in a new YANG module, as shown in Example 3-43.
Example 3-43 Deviation Module for ietf-interfaces.yang
module ietf-interfaces-deviations {
yang-version 1.1;
namespace "https://example.org/ns/ietf-interfaces-deviations";
prefix ietf-interfaces-deviations;
import ietf-interfaces {
prefix if;
revision-date 2018-02-20; // The NMDA version of ietf-interfaces
}
deviation /if:interfaces/if:interface/if:last-change {
deviate not-supported;
}
}
Note, however, that simply declaring a deviation does not make client applications understand your deviated model. Most likely, many client applications will fail to correctly interface with your system if you introduce deviations, so declaring a deviation is no good substitute for a correct and full implementation.
A server that implements a standard YANG module with deviations is not implementing that standard module, even if deviations were properly declared.
On rare occasions, engineers have been found to edit standard modules, rather than use deviation statements, in order to remove or change some content. That is a really bad idea. In most jurisdictions, that would be a copyright infringement and punishable by law. To misrepresent the contents of a module without changing the namespace is also likely to cause a lot of headaches for clients.
The deviation statement allows for other changes than just not-supported. It is possible to declare that a certain leaf uses a different type, for example, or uses a different default. Such deviations are much harder for clients to support, and therefore have limited value. If the purpose of declaring a deviation is to enhance the chance of interoperability with clients, those chances are pretty bleak if you go beyond not-supported, and not great even if you limit yourself to using only that.
Network Management Datastore Architecture (NMDA)
Most of the time when the operations engineer makes a configuration change and commits, the change takes effect immediately. Well, within few seconds maybe, depending on the change and the type of device. Some changes may not take effect immediately, however, because they depend on some required hardware that is not currently installed, for example.
Some NETCONF servers outright reject configurations that cannot be implemented immediately, but many accept configurations that are inherently valid, just not implementable at this time because of some missing dependency. They allow this because it is a very useful feature, often referred to as pre-provisioning or preconfiguration. The operator can preconfigure a feature, and the minute the right hardware is installed, the feature kicks into action.
With this background, it is easy to see that some configuration objects will not be found in the operational view of the system, at least not immediately and possibly never. Who knows what the future has in store? Similarly, some operational objects may exist despite not being configured yet. This is true in general, but let’s take a closer look at network interface management as an example, as shown in Table 3-4.
Table 3-4 Interfaces with Different Configuration and Operational States
Interface |
Configuration View (config true) |
Operational View (config false) |
|---|---|---|
GigEth1/1 |
Configured |
Absent due to missing hardware |
GigEth2/2 |
Configured |
Running as configured |
Loopback0 |
Absent |
Running with factory default settings |
Some device might have a network interface called GigEth1/1 in the configuration, but this interface does not exist in the operational state because there is no line card in slot 1 right now. There might be another interface called GigEth2/2 in the configuration. The hardware for this interface is present and the interface is running at full speed. Then there might be an interface called Loopback0. It was never configured, so it is absent in the configuration, but exists operationally, operating under some default settings.
Remembering the operators’ strong emphasis on the importance of separating configuration data from operational data in the requirements list in RFC 3535, the IETF working group invented the config true and config false concepts in the YANG language. Since a YANG list has to be either config true or config false, it’s not possible to have a single interface list that contains the sum of all the interfaces. In Table 3-4, the configuration view contains only two of three interfaces, and so does the operational view.
To accommodate the need for both types of information, the working group made sure all the standard YANG modules have a two lists—one for the configuration objects (for example, interfaces) and the other for the operational objects (for example, interfaces-state). The old standard ietf-interfaces.yang module (RFC 7223) is shown in Example 3-44.
Example 3-44 A Tree Representation of ietf-interfaces.yang in Its Original Form (Abridged)
module: ietf-interfaces
+--rw interfaces
| +--rw interface* [name]
| +--rw name string
| +--rw description? string
| +--rw type identityref
| +--rw enabled? boolean
+--ro interfaces-state
+--ro interface* [name]
+--ro name string
+--ro type identityref
+--ro oper-status enumeration
+--ro last-change? yang:date-and-time
+--ro phys-address? yang:phys-address
+--ro higher-layer-if* interface-state-ref
+--ro speed? yang:gauge64
This approach worked fine and was used for many years. Over time, however, more and more people started to realize there is a problem with this style of modeling.
As you might expect, most interfaces are present in both lists. They are both configured and have an operational state. The missing piece is how to navigate between the two. If you configured /interfaces/interface[name=‘GigEth2/2’], how would you know where the operational data for this interface lives? At this point, you might glance at Example 3-44 and sneer that it’s obviously /interfaces/interface-state[name=‘GigEth2/2’]. And you’d be right.
The question is how you teach a computer to do what you just did. You do not want to hardcode a lot of information into your NMS systems about /interfaces/interface being linked to /interfaces/interface-state. This knowledge must be built in directly in the protocol (for example, NETCONF), language (YANG), and modules (for example, ietf-interfaces.yang), in order to be automatically discoverable.
Note
Similarly, taking an example from the SNMP world, hardcoding in the NMS that the ifAdmin-Status and ifOperstatus Management Information Base (MIB) objects from RFC 2863 represent the read-write (the desired state of the interface) and read-only (the current operational state of the interface) views of the same object is costly–and most importantly, hardcoding every such piece of information does not scale.
Someone might suggest (and someone did) that the NMS simply rely on the naming convention that any configuration list called xyz would have an operational list called xyz-state next to it. While elegant in its simplicity, at least in this simple case, this idea was dismissed. Talk to your favorite software architect, and you’ll hear that reliance on naming conventions tends to go downhill in the long run. Every single YANG module would have to follow this rule, which would be difficult to enforce. And what seemed a simple rule turned out quite strange in certain cases.
Another disadvantage with the previously mentioned approach to modeling configuration and operational state separately is that there is a fair amount of duplication. There are two containers and lists for interfaces, and the keys (the interface name) are modeled twice. In this case, the interface type is also duplicated.
Combined with some new ideas for how to represent new forms of information that advanced servers might want to expose (for example, to support macros, system defaults, and sources of configuration other than operators and managers), the idea of extending the NETCONF datastore concept won strong support.
Instead of modeling the configuration and operational data separately, with a few small adjustments to a few RFCs, they could use a single, unified model. The client would just specify which datastore it is looking at as well as the running configuration or the operational one. Thus, the Network Management Datastore Architecture (NMDA, RFC 8342) was born.
With the revised NMDA model, the data objects are defined only once in the YANG schema, but independent instantiations can appear in different datastores (for example, one for a configured value and another for an operationally used value). This provides a more elegant and simpler solution to the problem. The downside is that many existing models would have to be redesigned. This is clearly a major drawback, even if the redesign is fairly natural and mechanical. This work is currently in progress at IETF. The reworked ietf-interfaces.yang module (RFC 8343), following NMDA, is shown in Example 3-45.
Example 3-45 A Tree Representation of the NMDA-Compliant ietf-interfaces.yang (Abridged)
module: ietf-interfaces +--rw interfaces | +--rw interface* [name] | +--rw name string | +--rw description? string | +--rw type identityref | +--rw enabled? boolean | +--ro oper-status enumeration | +--ro last-change? yang:date-and-time | +--ro phys-address? yang:phys-address | +--ro higher-layer-if* interface-ref | +--ro lower-layer-if* interface-ref | +--ro speed? yang:gauge64
Notice that the complexity and length of the model decreased significantly. Since there is only one list for all data pertaining to all interfaces, there is no longer any need for naming conventions or other crutches for finding all the pieces of information pertaining to a given interface.
What about the new datastores then? NETCONF initially specified three datastores, as shown in Figure 3-1.
:startup(optional):candidate(optional):running(always present)
Those three datastores map a typical router architecture, which contains some combination of a startup configuration, a running configuration, and candidate configuration.
Note
Capabilities in NETCONF have very long and unwieldy formal names, with lots of colons. The datastores are capabilities and therefore are afflicted by this as well. It isn’t surprising, then, that people started abbreviating the names, keeping only a colon (to indicate it’s an abbreviation) and the last word when referring to them. The :startup datastore’s formal name is urn:ietf:params:netconf:capability:startup:1.0.
With the NMDA architecture, additional datastores with described semantics may be defined for systems that support more advanced processing chains converting configuration to operational state. For example, some systems support configuration that is not currently used (so-called “inactive configuration”) or they support configuration templates that are used to expand configuration data via a common template. Such specifications may come from IETF or any other party. The NMDA specification itself took the opportunity to define two:
:intended:operational
Let’s examine the NMDA specification (RFC 8342) for the details regarding the content of each datastore.
The :operational datastore is a kind of old concept from the NETCONF world, now dressed up as a datastore; it holds all config false data and a read-only copy of all the config true data. It was just never officially named a datastore before. Then there is the brand-new :intended datastore. In simple servers, this one contains the same information as :running. In more advanced servers that might support a service concept, templates, and inactive (commented-out) configuration, the :running datastore contains the input data to services, templates, and inactive configuration, while :intended contains the result when those services and template constructs are expanded to concrete configuration elements, with inactive configuration removed. The :intended datastore is a read-only datastore, but it may still be useful to see what all those services, templates, and inactive elements expanded to.
The :operational datastore then contains all the configuration elements actually in use right now. No preconfigured elements. It also contains all the operationally active elements—even those that were not configured.
Interview with the Expert
Q&A with Martin Björklund
Martin Björklund is the editor (main contact) of the IETF YANG specifications RFC 6020, and later RFC 7950, and is also deeply involved in many other related documents. Martin co-founded Tail-f Systems in 2005, which was acquired by Cisco just under ten years later. Martin still works with the Tail-f products inside Cisco as architect, programmer, and distinguished engineer. Martin has a past as a serial entrepreneur, with a number of projects that were acquired or resulted in well-known companies.
Question:
What gave you and the original participants the audacity to design a yet another data modeling language and think you could change the world this way?
Answer:
(Laughter) We really hadn’t thought that it would receive the uptake we’re seeing now. We were completely focused on solving the NETCONF modeling problem. Well, actually, not only NETCONF. We already had our own proprietary language that applied to all management interfaces. That was always how we wanted to do this. Then there were political reasons for saying that this language was only for NETCONF. There had been several failed attempts at making modeling languages, not least at IETF. So trying to make a language that was good for everything was not popular. Don’t boil the ocean. Therefore, we kept limiting ourselves to NETCONF. That really worked well as a tactic to get it done, but later we have occasionally had to pay the price when obvious shortcomings for the more general case have showed up.
Question:
How did the cross-industry team behind this form in the first place?
Answer:
It started with general talk around mid 2006. The talk was that maybe something ought to be done, but that it would be hard. Everybody had different ideas. Some wanted to use XSD, some RelaxNG. Some thought there was no need for a language at all—why would it make sense to standardize the modeling language? At that time everybody had their own modeling language. We had ours, Ericsson had theirs, Juniper had their own. At IETF in Prague I spoke with Ericsson about this, then I shared a cab back to the airport with Phil Shafer at Juniper. That’s when we decided to give it a go. So it was that Phil Shafer, David Partain, Balász Lengyel, Jürgen Schönwälder and myself started this work. Later Andy Bierman joined the design team.
Question:
What were the main sources of influence for YANG?
Answer:
The point was that everybody had their own proprietary language. We had ours. Juniper had something. Ericsson had theirs. Jürgen was part of the SMIng design team at IETF, which never got around to be used. YANG started out with the concepts from those. Mostly ours and Juniper’s, with some SMIng factored in. The whole type system comes from SMIng, more or less. We took the best parts of each language and melded them together.
Question:
Did that help to make it acceptable among all the participants, since everybody contributed something?
Answer:
Yes, that definitely helped. That everybody would be writing YANG specs was hard to imagine back then. For a long time, nothing seemed to happen in the industry, while we kept building our tools. We got a head start of several years. Finally, people saw what it could do.
Question:
Ten years later, are you satisfied with the result?
Answer:
Yes, I have to say yes. Even if there are things that could have been done better. Technical details that should have been different. Definitely. Still, one has to be content with the end result. I think it’s good.
Question:
What’s so good with YANG compared to earlier languages?
Answer:
One cornerstone is readability. Most people that get a YANG document in front of them understand. People intuitively understand. Many other languages are much harder to grasp. It’s a pretty natural way to model hierarchical structures. Of course there are some constructs that aren’t easy, but overall, it’s easy to read. Fairly compact. The other very important thing is the extensibility. The way augment works, which allows amalgamating models from different sources and times. Augment allows you to extend existing models without changing the original, or even having to plan for extension, all the while keeping good backwards compatibility. There really aren’t any other languages that do this.
Question:
Is YANG finished now, or what happens?
Answer:
Yes, I hope it’s done now. The more popular the language gets, the more people are drawn in to contribute with their use cases and ideas. Wouldn’t it be nice if we could add ... for example, making YANG more object oriented. Many of these ideas don’t really fit into the YANG concept, and with every big change or addition, you always jeopardize the language. A version 2 or 3 of a language often kills it. Stability is a key feature of a language.
I hope the community will add things with some caution. No dramatic changes. We’re at YANG 1.1 now. YANG 1.2 with some fixes would probably be a good thing. Unfortunately, the most problematic issues with YANG right now aren’t small. The most annoying thing, in my mind, is that we have context-dependent values. Values that are instance identifiers, identities and XPath expressions are encoded differently for different YANG-based protocols. It’s not good when the values are coded using XML namespaces in XML, when you use JSON, which doesn’t have that. To update YANG to fix this in a good way won’t be easy, however.
Summary
This chapter started off with showing you how to take some real-world data and turn that into a formal YANG contract describing the type and structure of that data. This allows clients to know what to expect, and servers to implement industry standards. Model-driven APIs are essential in a software-defined world.
Then the modeling turned to more complex features in order to mirror real-world data organization. For example, tables in tables, keys, and references—as well as how to model data where the set of acceptable values grows over time. Extensibility is a key feature of YANG, both with respect to the values, the elements, and the YANG language itself.
Traditionally, part of the management interface that was the hardest to get right was configuration. Much attention was therefore given to the strong consistency mechanisms in YANG, such as must and when expressions. Tight and specific YANG contracts make the models dependable and useful. The strong separation between configuration and operational data must also be counted as a consistency feature.
After the configuration discussion, action, rpc, and notification constructs were used to define client/server communication beyond the configuration transactions and operational status collection.
The section towards the end on pointers and XPath explained (and hopefully prevents) some of the most common mistakes that crop up repeatedly in YANG Doctor reviews in recent years.
At the end of the chapter was a discussion around the background and implications of the now ongoing redesign of essentially all the standard IETF YANG models, to adhere to the principles of the Network Management Datastore Architecture (NMDA). This shift allows simpler and more compact models, while also giving greater flexibility for clients to work more closely with servers that implement advanced features like macro expansion, data learned from the network, and pre-provisioning.
References in This Chapter
This chapter is by no means a complete account of everything you can do in YANG, but it aims at providing you a good grasp over all the most important concepts, what problems YANG solves, and how to go about making new YANG models.
To go deeper, the next step is the RFCs. They are actually quite readable and a good source of detailed information, and many have usage examples. In particular, have a look at the references in Table 3-5.
Table 3-5 YANG-Related Documents for Further Reading
Topic |
Content |
|---|---|
YANG 1.1 YANG 1.0 |
https://tools.ietf.org/html/rfc7950 https://tools.ietf.org/html/rfc6020 YANG Data Modeling Language is defined in RFC 7950 (YANG 1.1) and RFC 6020 (YANG 1.0). Defines all YANG keywords and the mapping to NETCONF. Provides some examples. |
XPath |
https://www.w3.org/TR/1999/REC-xpath-19991116/ XML Path Language (XPath) Version 1.0 is the XPath version used in YANG. It is defined by the World Wide Web Consortium (W3C). Defines the XPath language used inside YANG |
REGEX |
https://www.w3.org/TR/2004/REC-xmlschema-2-20041028/ XML Schema Part 2: Datatypes Second Edition defines the flavor of regular expressions used in YANG YANG |
NMDA |
https://tools.ietf.org/html/rfc8342 Network Management Datastore Architecture is defined in RFC 8342. This is the new recommended way of structuring YANG modules. |
YANG Guidelines |
https://tools.ietf.org/html/rfc8407 Guidelines for Authors and Reviewers of YANG Data Model Documents are defined in RFC 8407. This is highly recommended reading before reviewing or writing YANG modules. Contains conventions and checklists. |
NETMOD WG |
https://datatracker.ietf.org/wg/netmod/documents/ IETF Network Modeling Working Group. This is the group that defines YANG and many of the YANG modules. Look here for the latest drafts and RFCs. |
IETF YANG GitHub |
https://github.com/YangModels/yang IETF’s repository of IETF, other SDOs, open source, and vendor-specific YANG models. Good place to take inventory, but necessarily incomplete and often outdated. About 10,000 YANG modules are checked in at the time of writing. |
|
Catalog of YANG modules, searchable on keywords and metadata. Also has YANG tools for validation, browsing, dependency graphs, and REGEX validation. At the time of writing there are about 3500 YANG modules in the catalog. |