Chapter 3

Equations

3.1. Introduction

OUR purpose here is to give some applications for each of the five basic types of equations (and of their hybrids). These types are: algebraic, differential (ordinary and partial), difference, integral, and functional. Diophantine equations, algebraic equations with rational coefficients whose solutions must be integers, are also illustrated. Each of the above types of equations may also have stochastic elements; this type of problem often arises in modeling. We do not illustrate these extensively in this chapter. Hybrid equations include differential–difference, integro-differential, integro-difference, integro-differential–difference, etc.

There are no clear dividing lines between the examples in this chapter and those in others, especially since equations and inequalities are so central to all quantitative mathematics. However, the purpose of this chapter is to catalogue important types of equations. Few of the equations are solved; we are concerned essentially with their formulation. A comprehensive treatment of most types of equations is given in Nonlinear Mathematics by Saaty and Bram and Modern Nonlinear Equations by Saaty included in the bibliography (now to appear in paperback by Dover Publications).

3.2. Algebraic Equations

Very briefly an equation of the form

where a 0 , a 1 , a 2 ,…, a n are real or complex numbers, is called an algebraic equation of degree n . We know from the fundamental theorem of algebra (due to Gauss) that for every algebraic equation of degree n there exists n roots or solutions .

Frequently we are interested not in a single equation but in a system of equations. Among the simplest systems of algebraic equations is a linear system of m equations and n unknowns x 1 , x 2 ,…, x n given by

In algebra the existence of a solution of such a system is given by means of the concept of the rank of the matrix of coefficients and the rank of the augmented matrix which includes the right side constants as an additional column. This matrix is not square. The details would take us far afield, but this system is solvable if and only if the rank of the matrix of the system is equal to the rank of the augmented matrix. A system of linear equations is said to be overdetermined if the number of variables n is less than the number of linearly independent equations m . It is determinate if m = n and underdetermined if m < n.

If b i = 0 (i = 1…m ), then we have a system of m homogeneous equations in n unknowns. If m = n , this system has a nontrivial solution if and only if the determinant of the matrix of the system is equal to zero.

Systems of nonlinear algebraic equations are usually solved by numerical approximation methods such as Newton's method and gradient methods.

We give a simple example which involves algebraic equations.

Example . Strategy for a Mailman

A mailman must deliver mail to each house on both sides of a straight street of length L and width W. Suppose that all N houses have the same street frontage of length D. The houses on both sides may be considered as points in the plane and start precisely at the beginning of the street and end at its end. The houses on one side are a mirror image of those on the other. If the mailman crosses the street he does so on a perpendicular to its length. Compare the strategy of delivering mail to all houses on one side, crossing the street and returning to his starting point by delivering to all houses on the other side, with the strategy of crossing the street from one house to that opposite it on the other side, walking to its next door neighbor and crossing the street again.

We want conditions on D and W such that the mailman would follow the first or second strategy to walk the least distance. Using the first strategy he walks 2L + W and, using the second, he walks (N /2) W + L .

The problem is: when is 2L + W < L + (N /2) W which is equivalent to L < [(N /2) – 1] W . Substituting L = [(N /2) – 1] D gives [(N /2) – 1]D < [(N /2)– 1] W or D < W . Thus, he uses the first strategy when D < W and the second when D ≥ W .

3.3. Diophantine Algebraic Equations

Sometimes the solutions of an algebraic equation must be integers. A simple example follows.

Example. Economic Constituency of an Air Fleet

An airline buys Boeing 707 aircraft at $12 × 106 each, Boeing 727s at $8 × 106 each, and small executive type jets at $2 × 106 each. If a total sum of $8 × 107 is paid for 20 aircraft, including at least one of each kind, how many aircraft of each kind does the airline buy?

If x , y , z are the respective numbers bought of each kind of aircraft mentioned in the problem, then one must find positive integers x , y , z which satisfy x + y + z = 20, 6x + 4y + z = 40. Subtraction gives 5x + 3y = 20. For this equation to hold, y must be divisible by 5. Thus we begin by choosing y = 5 from which we get x = 1 and z = 14. In fact this is the only possible solution.

3.4. Ordinary Differential Equations

Recall that a differential equation is an equation involving functions of one or more independent variables and certain of their derivatives. Such an equation is called ordinary if there is one independent variable. The order of a differential equation is the order of the highest order derivative it contains.

A functional relation between the dependent and independent variables which satisfies the differential equation is said to be a solution of the equation. A differential equation is completely solved when all of its solutions are known.

If m differential equations are given in n unknown functions, we speak of a system of differential equations.

By a solution of the system of equations of the first order

we mean a system of functions

which satisfies the system simultaneously.

Given this system and a point P (a , b 1 ,…, b n ), if the functions f 1 , f 2 ,…, f n are continuous in a neighborhood N (E ) of P and have continuous partial derivatives there with respect to the variables y 1 , y 2 ,…, y n , then in a certain neighborhood of a there exists precisely one set of functions which is a solution of the system satisfying the initial conditions

Sometimes boundary conditions are imposed on the solution leading to a boundary-value problem. In general, the number of equations need not be equal to the number of unknowns. Under such general circumstances, the conditions given above on the existence and uniqueness of the solution need no longer be valid.

We now give some illustrations of models in which ordinary differential equations are used.

Example 1. Advertising

When two competitive companies engage in advertising campaigns, it is reasonable to assume that the rate at which each company increases its sales is proportional to the available market. (The constants of proportionality will depend on the effectiveness of each campaign.) Thus we have the following differential equations:

where S i (t ) is the sales of company i in any year (S /year), M a (t ) is the available market = M (t ) – S 1 (t ) – S 2 (t ), M (t ) is the total market, and C 1 and C 2 are constants.

The solutions for S 1 (t ) and S 2 (t ) will depend on the form for M (t ) which on occasion has been taken to be

where α and β are constants.

We have,

Now the first-order linear differential equation

has the general solution

Thus the general form of the solution for S 1 (t ) is given by

A similar expression may be obtained by the reader for S 2 (t ).

Example 2. Lanchester's War Model

The following differential equation model, developed by Lanchester, is concerned with the problem: Suppose that N 1 units of one force A , each of hitting power α , are engaged with N 2 units of an enemy B , each of hitting power β . Suppose further that the engagement is such that the fire power of force A is directed equally against all units of B and vice versa. The rate of loss of the two forces is given by

where k 1 , k 2 are positive constants. (N 1 and N 2 are, of course, integers, but it is convenient to treat them here as continuous variables.) The strength of the two forces is defined as equal when their fractional losses are equal, that is, when

This type of analysis is used in balance of power problems. On dividing the first equation by the second and integrating, we obtain two hyperbolas defined by αN 1 2 – βN 2 2 = C . Taking C = 0 gives Lanchester's N 2 law, which states that the strength of a force is proportional to the fire power of a unit multiplied by the square of the number of units.

3.5. Partial Differential Equations

An equation of the form

which relates the unknown function z (x 1 , x 2 ,…, x n ), (n ≥ 2) and its derivatives is called a partial differential equation. The order of the highest order derivative appearing in the equation is called the order of the equation. In general, we may consider a system of partial differential equations for the unknown functions z 1 , z 2 ,…, z r . If the number of equations is greater than the number of functions to be found, then, in general, there is no system of functions to satisfy all the equations. In this case, the equations are said to be incompatible. Certain conditions, the so-called conditions of integrability, must be satisfied in order for the solution to exist. For example, a necessary condition for the system

with z ≡ z (x , y ) to be solvable is that the mixed second partial derivatives ∂ 2 z /∂y∂x and ∂ 2 z /∂x∂y should be equal. Now

If we impose equality and substitute

we obtain the necessary condition

(A , B , ∂A /∂y , ∂A /∂z , ∂B /∂x , ∂B /∂z are assumed to be continuous.)

We give one example here. Other examples can be found in later chapters.

*Example. Spending Money for the Taxi Driver's Wife

Every time a taxi driver earns a total sum of μ dollars, he gives it to his wife. The time for earning this sum is a random variable, exponentially distributed with mean 1/λ . His wife spends a dollar per day, but only as long as she has money. She does not accumulate any debts. Let (F (x , t ) = prob [the wife has an amount of x or less dollars at time t ]; determine F (x , t ).

Clearly, F (x , t ) = 0 for x < 0.

Since the time to earn μ dollars is exponentially distributed,

By Taylor's expansion,

Substituting in (3.1 ) and (3.2 ) yields

By letting h → 0 we obtain the equations

which may then be solved to find F (x , t ).

3.6. Difference Equations

An ordinary difference equation of the k th order has the form

where x is a real variable which depends on an integer valued variable n and k is an integral increment added to n . As with differential equations, the solution of a difference equation requires boundary conditions to ensure uniqueness.

The above form may be extended to the case where n is replaced by a real continuous variable t but k remains an integer. Thus, we write x (t + k ) = f [t , x (t ), x (t + 1),…, x (t + k – 1)]. A system of simultaneous difference equations may take the form

Such a system may sometimes be reduced by elimination to a single equation. The number of equations can be increased by replacing n by n + 1, for example, and repeating this procedure in order to obtain a sufficiently expanded system to enable elimination of all the y 's. Problems which seem to resist easy formulation as algebraic equations may often yield nicely to difference equation methods. We give a simple example. It is taken from real life.

Example. The Fish Aquarium

A fish aquarium contains n units of water. Each week one unit evaporates and must be replaced with fresh water. Because fresh water contains uniformly a certain amount of salt, there is a possibility that the concentration of salt in the aquarium will become dangerously high for the fish. To cut down this risk, at the end of the week, when n – 1 units are present, another unit is removed from the aquarium (leaving n – 2 behind) and then two units of fresh water are added. This will still lead to an increase in the concentration of salt, but not as much as if the additional unit were not removed. Prove that ultimately the concentration of salt per unit in the aquarium approaches twice that in fresh water.

Let c be the concentration of salt per unit of fresh water and let X k be the total amount of salt in the aquarium at the end of the k th week with the level of water brought to normal. We have

from which it may be seen that the concentration approaches 2c .

3.7. Differential Difference Equations

We now consider equations in which there are both derivatives and differences; we illustrate with a simple example.

Example . Cascading Cups [ American Mathematical Monthly ]

A series of cups of equal capacity have been filled with water and arranged one below another. Pour into the first cup a quantity of wine equal to the capacity of the cup at a constant rate and let the overflow in each cup go into the cup just below. Assuming that complete mixing of wine and water takes place instantaneously, find the amount of wine in each cup at any time t and at the end of the process at time T .

Let q denote the capacity of each cup, so that the rate of flow is q /T .

Let x k be the amount of wine in the k th cup and x k - 1 that in the (k – 1)st cup. Then the rate of flow of wine into the k th cup is x k - 1 /T and out of it is x k /T . This gives

Any equation in the sequence can be solved if the one before it has been solved. Thus

and so on.

For the final amounts of wine in the successive cups one has

where e k is the sum of the first k terms of

(Note that as k → ∞, x k → 0.)

It is not essential that the rate of flow be constant. If x , the amount of wine poured into the first cup, is the independent variable, then

The final result (at time T ) is obtained by setting x = q .

3.8. Delay–Differential Equations

Differential equations with deviating arguments in which the arguments of the highest order derivative of the unknown function are identical and assume values that are never less (more) than those of the unknown function and its remaining derivatives are called equations of delay, retarded , or lagging (advanced or leading) type. We illustrate with an example.

Example . Population Growth—Delayed Harvest

Consider the delay–differential equation

The left side has a form identical to that of the strength of a force in Lanchester's equation. This equation can be used to describe the net birthrate of a population z (u ) at time u , and expresses the condition that this birthrate is a constant minus a quantity proportional to the population at a previous time u – r .

If we put u = rt , multiply both sides of the equation by r 2 β , and define x (t ) ≡ βrz (rt ), the equation now becomes

where a = αr , which is a slightly simpler form of a delay–differential equation.

3.9. Differential–Difference–Delay Equations

We now give an example where the equation which is obtained involves derivatives, differentials, and delay in the arguments.

Example Car Following [L. C. Edie.]

Here we describe the simplest situation of identical car following, such as might occur on long stretches of highway in dense traffic when no passing is possible. An approximate description of the way in which a car follows a leader is given by the relative velocity control in which acceleration at time t of the following car is proportional to the relative velocity of the two cars at a retarded time t – ▽ and inversely proportional to the distance between them. This model has been tested in practice and found to be satisfactory.

We may express these conditions as follows:

where M is the mass of car, x n (t ) is the position of car n at time t , ∆ is the lag time of system, and λ is the driver–car sensitivity coefficient (constant).

We use the convention that ẋ n (t ), ẍ n (t ) denote that one should differentiate once and twice with respect to time and hence denote velocity and acceleration respectively.

We note that this equation may be interpreted both as a dynamic equation of motion and as a stimulus–response equation. If we integrate the equation, we obtain

where L is the length of each car.

3.10. Integral Equations

There are also models in which the relation to be satisfied involves an integral in an unknown function (which has to be determined).

*Example. Renewals [Saaty, 1961]

We consider a system with many components. We know from experimental results that the probability of failure of an important component is f (x )dx during an interval of time dx . There are N of these particular components in the system, and we need to know the expected number of renewals which will be needed by time T .

We define renewals of the first kind to be renewals of the initial components, renewals of the second kind to be renewals of renewals of the first kind, etc. Then if u i (t ) is the expected number of renewals of the i th kind at time t , we have

We define

Then the expected total number of renewals U (t ) by time T is given by

where

3.11. Integro–Difference and Integro–Differential Equations

We now consider some examples in which the equations which we obtain contain both integrals and differences or derivatives.

*Example 1. A Queue

Suppose that customers arrive before a single server and queue up to be served on a first-come, first-served basis. This is known as ordered service. The waiting time (when there is a line) of the (n + 1)st customer w n + 1 equals the waiting time of the n th customer w n plus his service time s n , minus the time between the arrival of the n th and (n + 1)st customer which we denote by t n .

Thus

Since each of s n and t n is given by a probability distribution so are w n and w n + 1 . We wish to determine these unknown distributions.

To include the possibility of no waiting, we write w n + 1 = max(w n + s n – t n , 0). We wish to calculate the waiting-time distribution of a customer. Let us suppose that the (n + 1)st customer will wait. We seek

Denote by P n + 1 (w ) and P n (w ) the distributions of the waiting times in the queue of the (n + 1)st and n th unit. We assume that the variables t n and s n are independently distributed according to the distribution functions A (x ) and B (y ), and that the waiting time of the n th unit is independent of its own service time and of the interarrival time of the (n + 1)st unit.

Accordingly, the probability that the waiting time w n satisfies the above relation and that the interarrival time t n and the service time s n satisfy

is

This formula may also be obtained by considering the distribution of the sum of three independently distributed random variables. In the steady state the probabilities are independent of time and of the initial state of the system and all units have the same waiting time distribution P (w ). Hence

Example 2 . Conflicting Populations

We now consider two populations in which a predator–prey situation exists. The rate of change of each population is assumed to be proportional to its present size, to the number of its encounters with the other population, and to a term which reflects the way in which the past history of the other population affects the present size of the given population. This yields the following equations for what has become known as a Volterra system.

N 1 (t ) and N 2 (t ) are the numbers in populations 1 and 2; the remaining coefficients are positive constants.

* 3.12. Stochastic Differential Equations

It is convenient to distinguish three basic types of stochastic differential equations. Such equations may involve either:

(i) random initial conditions;

(ii) random forcing functions; or

(iii) random coefficients.

In the first case, the given probabilistic properties of the initial conditions (regarded as random variables) determine those of the solution x (t ) through the explicit form of x (t ). A typical example is when x (t ) represents the motion of a particle subject to uncertainty only in its initial position and governed by deterministic forces. See Chapter 8 by R. Syski in Saaty, Modern Nonlinear Equations.

In the second case, the stochastic process representing random forcing functions is given, and the properties of the solution x (t ) are developed through probabilistic argument. A typical example arises in the study of the output of a physical system when the input is a Brownian-motion process.

Finally, in the third case, the parameters of the system are regarded as the random variables, and the stochastic properties of x (t ) reflect the influence of randomness imposed by the structure of the system. For example, the output of an electric circuit with randomly varying capacity is described by equations of this type.

We consider situations in which each type of equation might occur rather than taking very specific examples as in the earlier sections. Our examples may most easily be applied in dynamics, but there are many other problems where these conditions apply.

Example 1. Random Initial Conditions

Consider simple linear motion. This may be described by an equation of the form , where x (t ) = position at time t and c = constant. We assume c ≥ 0.

The solution is given by

where x 0 gives the position at time t = 0.

Suppose that x 0 is a random variable with a given distribution. The mean, E {x (t )} and the variance, var{x (t )}, of the random variable x (t ), can be calculated directly, even without the explicit form of this distribution. We obtain

Suppose now that x 0 has a Gaussian distribution with mean zero and variance σ 2 . Then the probability that x 0 assumes values between λ and λ + dλ is given by f 0 (λ )dλ , where the density f 0 (λ ) has the form

We may now find the distribution of x (t ).

Thus, x (t ) has a Gaussian density f (α , t ), where

which has the same variance σ 2 as x 0 but which has now a mean of ct.

Example 2. Random Forcing Functions

We are given a stochastic process (or a random function) z = {z (t ), t ≥ 0} with specified stochastic properties. The differential equation, which we wish to consider, relates an unknown random function x = {x (t ), t ≥ 0} and its derivatives to the given random function z . The solution of the equation will then define a new stochastic process {x (t ), ≥ 0} in terms of the given z process.

Because of the many physical applications, the function z (t ) is known as a forcing function.

To illustrate the problem, we consider the simple case of the first-order equation

The solution is given by

It can be seen that x (t ) depends on the values of z (τ) over the interval 0, t , and it is not immediately clear whether the integral above defines a random variable. This brings up the question of what is meant by a stochastic integral.

A related problem is the meaning attached to the stochastic differentiability of x (t ); questions of existence and uniqueness of stochastic differential equations should then be examined. A point of view helpful in these considerations is to regard a random function, say x , as a function on the real line to the space of random variables. Thus, the value of x at t is a random variable x (t , w ), a point in a function space of random variables. Problems of continuity, differentiability, and integrability of random functions now become the same problems as those for ordinary functions which take values in abstract space; the same comment applies to differential equations.

Furthermore, explicit expressions for the distribution (or its transform) of x (t ) are required in terms of those for z (t ); this task, important for applications, is seldom easy. However, under suitable conditions, integration and averaging commute so that in the above example,

Example 3. Random Coefficients

We consider the well-known application of damped simple harmonic motion

Suppose that the coefficients a and b have a joint uniform distribution over the square – 1 ≤ α ≤ 1, –1 ≤ β ≤ 1 with density 1/4. We need to find the probability that every solution x (t ) tends to zero as t → ∞ (i.e., the solution is stable). This occurs when zeros of the random characteristic equation

are either (i) both real and both negative, or (ii) both complex with negative real parts. These two disjoint events may be described by random variables as follows. If a 2 – b ≥ 0, then both roots are real, and they have the same sign if and only if b > 0, this sign being negative if and only if a > 0. If a 2 – b < 0, then the roots are complex conjugate, and they have negative real parts if and only if a > 0; in this case also b > a 2 > 0. Hence, the required event will occur if and only if a > 0 and b > 0. The probability that this will occur is:

Note also that the probability that the zeros of the characteristic equation are complex is

3.13. Functional Equations

Generally, equations involving implicit relations between simple functions and functions of other functions are called functional equations. We give two simple examples.

Example 1. Total Purchase Expenditures

Consider a process which occurs in N stages. An example of this would be purchases over a period. Let x n be the input at stage n , d n be a decision or control variable at stage n (an example of a control variable is the amount purchased in a process at the n th stage), r n be a return function which depends on x n and d n , and suppose that the input between two consecutive stages satisfies the relation

Then the total purchase expenditure for (x n ) in n stages is given by an equation at the form

As n → ∞ this equation becomes

which is a functional equation from which f (x ) must be determined.

*Example 2. Information Theory

From the standpoint of information or knowledge, the presence of a problem is a state of uncertainty. Solving the problem totally or partially removes or reduces this uncertainty. Thus, a problem may be first regarded in terms of the amount of uncertainty in it; logical operations and experimental tests may be used to gather information which decreases the uncertainty (to zero if possible). Uncertainty or entropy content (the negative of this quantity is defined as the information content) of a set of n mutually exclusive events is given by

where p i is the probability that the i th factor arises in defining the problem.

In the continuous case, the entropy is given by

The form which these expressions take arises from the requirement that the sum of the probabilities causing the occurrence of two independent events is equal to the probability of the occurrence of the two events together.

This yields a functional equation of the form

which has f (x ) = log x as a solution. The expected value gives the entropy. Recall that

depending on whether x is discrete or continuous. Thus we obtain

or

Chapter 3—Problems

1. Formulate the following problem algebraically. A band of 13 pirates obtained a certain number of gold coins. They tried to distribute them equitably but found that they had eight left. Two pirates died of smallpox. The remainder tried to distribute the coins equitably among the 11 remaining pirates, but they found that there were three left. Thereupon they shot three pirates, but still there was a remainder of five coins when they attempted an equal division among the eight remaining pirates. How many coins were there? (All numbers of the form 333 + 1144k .)

2. What number, when added to 100 and to 164 yields the square of an integer? Find all solutions? (125, –64, –100.)

3. Consider a river in which the velocity of the current is V in the direction shown by the arrow (Fig. A ). A man swims from A to B and back to A . Another man leaves A at the same time as the first, and, swimming at the same rate, swims to C and back to A . If the distances AB and AC are equal, AB being parallel to the current and AC perpendicular to it, which man gets back first? (Hint : Note that the man swimming towards C would have to aim for a point other than C , to allow for the effects of the current.)

FIG . A.

4. Suppose that the rate at which a rumor spreads varies directly with the number of people who have heard the rumor. Show that the rumor grows exponentially. Is this a good model? Is the assumption reasonable? Can you modify your equation to take care of possible problems?

5. Find P n , the probability that an event will occur on the n th trial, given λ , the probability of its occurrence on the n th trial if it occurred on the (n – 1)st trial and μ , the probability of its occurrence on the n th trial if it did not occur on the (n – 1)st trial. [p n = λp n –1 + μ (1 – p n –1 ).]

References

Cascading cups, Am. Math. Monthly , Vol. 28, 1921, p. 144.

Edie, L. C., Car-following and steady state theory for non-congested traffic, Operations Res. , Vol. 9, No. 1, 1961, pp. 66–76.

Bibliography

For a comprehensive list of references on equations and inequalities, consult:

Saaty, Thomas L. and J. Bram, Nonlinear Mathematics , McGraw-Hill, New York, 1964.

Saaty, Thomas L., Modern Nonlinear Equations , McGraw-Hill, New York, 1967.

Saaty, Thomas L., Elements of Queueing Theory, with Applications , McGraw-Hill, New York, 1961.

Saaty, Thomas L., Mathematical Methods of Operations Research , McGraw-Hill, New York, 1959.

Saaty, Thomas L., Optimization in Integers and Related Extremal Problems , McGraw-Hill, New York, 1970.