Monday, September 10, 2012

Linear Differential Equations III: Undetermined Coefficients


← Part II, The Homogeneous Case                           {Causality} Next →

Consider the linear differential equation of order n.
          
where an ≠ 0 and X(t) ≠ 0.

It has solution y = yc + yp.  The homogeneous solution yc can always be found in a straightforward manner.  We would like to find the particular solution, yp.  There are a number of methods, depending on the properties of X(t).   The simplest, to me, is

THE  METHOD OF UNDETERMINED COEFFICIENTS
 If X(t) is a linear combination of functions, and each term has a finite number of linearly independent derivatives, then yp is a linear combination of X(t) and its linearly independent derivatives.  The coefficients of each term  are unknown.  Substitute the trial solution into (1), and solve for the unknown coefficients.

Basically, we already know we're going to differentiate our solution a bunch of times.  n times, to be exact.  And when we're done, we need to be left with X(t).  So differentiate X(t) and see what functions are introduced.  Make up a solution which is stupidly  AllOfThoseFunctionsAddedTogether.  The only question left is then what constants to multiply them by, so we really are left with X(t).

That's it.  Really.

Example 1:
          
We have  X(t)  =  2t + ksin(t).   The elementary functions in X(t) are   t  and   sin(t).   The linearly independent derivatives are:
           t :      1
     sin(t):    cos(t)
Our trial solution  has the form:
          
Differentiating twice,
          
Substitute these values into the differential equation,
          
Collect like terms:
  
The coefficients on the left-hand side must match the ones on the right.
          
The rest is Algebra.
       
We now have the particular solution:
       
From the characteristic equation m2 - 2m + 3 has roots  m = 1± i√2, and the homogeneous solution is
       
At last we have the general solution,  :
       .


Example 2:

       
For periodic functions such as sine and cosine, another approach is available.  From the equality   ,  we know that  cos x  is the real part of eix.  Consider, then,
       
If Yp is a solution of (2a), then yp =     (the real part of yp) is a solution of (2).
A trial solution is then
       
Substitute into (2a):
       
And divide through by e0t,  which  is never ever ever zero, foreverEverPlus1, no takebacks.
       
The particular solution of (2) is then
     
In addition, the real part of any complex exponential can be written as a linear combination of two complex exponentials with equal exponents of opposite sign, which in turn can be written as a single sine or cosine function of the form     Asin(ω0 t + θ1)    or      Acos(ω0t −θ2).

In this example, construct the right triangle with base 3 and height 2ω0.  Call the angle adjacent to the base θ.  Then
       
And yp can be rewritten as
       
where   tanθ  = 2ω0/3.

Everything appears to be in order.  It is time to move on to an elementary example from signal processing.  I am not sure why people who work on signals insist on making things more complicated and confusing than necessary.  That is part of what I am unraveling as I catch up with the job I want.  Something is terribly wrong with the current state of audio tools and design.  Audio is full of crazy.   But I know how to tell the difference between a coat hanger and the ghost of my grandfather. So bring it on.

Bring on the crazy.

II. The Homogeneous Case

[Linear Differential Equations of Order n]
← Part I                                           Part III  →

II. THE HOMOGENEOUS CASE, CONSTANT COEFFICIENTS
If Q(x) = 0, the equation is homogeneous.
     (19.31)     

■ If the fk(x) are arbitrary functions of x, solutions of (19.31) are rarely expressible in terms of elementary functions.  When they are, it is generally extremely difficult to find them.  If the fk are constants, the general solution of (19.31) can always be found.

The homogeneous linear differential equation of order n with constant coefficients is thus
     
Now, the derivative of ex is ex.  Also, ex is never ever ever ever zero, so we can divide it out of any equation, for all x.  Using the Hey Wait a Minute... Postulate**, if we let  ,  then
          
Plugging this trial solution into (20.1) we  have
          
Divide through by emx,
      .
Hey, aren't those all constants?  Any value of m which satisfies this equation makes y a solution of (20.1).  Our machinations are full of win.

Definition: Equation (20.14) is the characteristic equation of (20.1).

Sooooooo..... this is a regular-ass algebraic polynomial in m.  It has at least one and not more than n distinct roots.  We have n solutions of the form   ,  with at least one, and not more than n, distinct values of n:
   
where the mk need not all be distinct.

We are confronted with three cases:
     1. All of the roots are real, and distinct
     2.  All roots real, some multiple
     3.  All roots imaginary
All other cases are linear combinations of these three.  The cases can be treated separately, and added together to form a composite solution yc.   (The proofs are in Part I).

CASE 1: Roots of the Characteristic Equation Real
If all n roots, m1, m2, . . . , mn  are distinct, (and extending the above proof), the n solutions  y1, y2, . . . , yare linearly independent.   The general solution of (20.1) is then
     

In this case, the procedure is straightforward.  Solve for the roots m, and construct the solution.  "Solve for the roots m" can be arbitrarily difficult, but it is at least a separate problem.  Rad.


CASE 2: Roots of the Characteristic Equation Real, Some Multiple
{Coming back to this, I have an elementary example of Case I to work out.  Simple but for the approach taken}

CASE 3:  Roots of the Characteristic Equation Imaginary
{To Do.}

__________________________
**Squint one eye, make a serious face, and put your hands on your hips.
At some point you may need to lean back and cross both arms over your chest.

Linear Differential Equations of Order n: I

Oppenheim is telling stories about linear differential equations; I review the material necessary to right myself.
Following  Tenenbaum and Pollard,

■ A linear differential equation of order n is an equation which can be written in the form
,
where f0(x), f1(x),  . . .  , fn(x) and Q(x) are continuous functions of x defined on a common interval I, and fn(x) is nonzero somewhere in I.   Note that y(k) is the kth derivative of y.  For example,  y''' = y(3).

Definition: Let the functions f1(x), f2(x),   . . .  , fn(x), be defined on a common interval I. Then the functions are linearly dependent  if there exist  constants c1, c2, . . . , cn, not all zero, such that

for every x in I.
     The functions are linearly independent if no such set of constants exists.


■ Theorem 19.3:  If f0(x), f1(x), . . . ,  fn(x) and Q(x) are continuous functions of x on a common interval I and fn(x) ≠ 0 when  x is in I, then
 1. The homogeneous linear differential equation
 
has n linearly independent solutions y1(x), y2(x), . . . , yn(x)
2. The linear combination of these n solutions
,
c1, c2, . . . , cn arbitrary constants,  is an n-parameter family of solutions of (a).
3. The function

where yp is a particular solution of the nonhomogeneous equation (with Q(x)  ≠ 0),  is an n-parameter family of solutions of (18.11).

(Chapter 19, pg. 211:)
"It is extremely important that you prove the statements in Exercises 5 to 7 below."

■  5. If yp is a solution of
     ,
then Ayp is a solution of (19.5) with Q(x) replaced by AQ(x).

Proof:
If yp is a solution of (19.5), we have
       
Mulitply through by A
       
And since
       
With the A's pulled into the derivatives, we have
       
And Ayp is a solution of (19.5) with Q(x) replaced by AQ(x).


■ 6. Principle of Superposition.  If yp1 is a solution of (19.5) with Q(x) replaced by Q1(x) and yp2 is a solution of (19.5) with Q(x) replaced by Q2(x), then yp = yp1+ yp2 is a solution of
          .
Proof:
Add the two equations in yp1 and yp2:

And since differentiation distributes:  (u' + v') = (u + v)',

After the substitution yp = [yp1+ yp2],
       
Done.

 7. If yp(x) =  u(x) + iv(x) is a solution of
      ,
where f0(x), . . . , fn(x) are real functions of x, then
     (a) the real part of yp, i.e. u(x), is a solution of
               ,
     (b) the imaginary part of yp, i.e. v(x), is a solution of
               .
Proof:
Writing yp(x) as u + iv,

differentiate the terms separately,

and collect real and imaginary parts on the left-hand side:

Two complex quantities are equal iff their real parts are equal and their complex parts are equal.  That is,
          
AND THE RIGHTEOUSNESS IS COMPLETE.

                             Next:  The Homogeneous Case   →
__________________________
Notes: To make the material easier for me to find, I use the numbering from Tenenbaum and Pollard's Ordinary Differential Equations, Dover Press.  If you find my personal math blog useful, I am happy to change the numbering and layout for easier navigation.  Let me know.

Thanks as always to CodeCogs for the Latex Equation Editor.