Spatialization IIa

                        Part II →

{Under heavy editing. Incomplete}

Start naive.  Build the problem incrementally.

Fig. 1

In Figure 1, source S moves along an arbitrary, directed path f(t).  Microphones L and R and implied listener O' may be placed however we wish.

PROPOSITION:

To construct an image-preserving mapping from S to O'.

INITIAL ASSUMPTIONS 
Arrange the listener and microphones in an equilateral triangle ΔLO'R:   .
All distances are in meters.
Let v_L[n] and v_R[n] be the voltage recordings at L and R, respectively:
     
   Alternately,
     
Assume the recordings are normalized:   .
Let
   ρ (rho) = sample rate of both recordings, in samples/second.
   ψ (psi) = the constant speed of sound in meters/second
    t    =      time in seconds

S is a physical object represented by a closed mesh.

Let
    
be the (magnitudes of) unknown disturbances propagating from S. Given initial time and orientation (t₀, θ₀, φ₀) the amplitude in an arbitrary direction in the future is determined by the time t, and two angles θ, φ.

Specifically, assume μ is the result of a distinct motion in space, constrained to a small neighborhood of a central point  m:(m₁, m₂, m₃) within the bounding box of S.  Further, assume μ(t, θ, φ) and point m imply one another.  We have the inseparable tuple   .
I make no such assumption about σ(t, θ, φ).  This distinction is essential.

Let
       
be a container for the image of S.  X is a spatial picture: a 3-D "speaker" aligned with S, broadcasting into the game world.

 The closure of the disturbance is the region enclosed by the maximum progression of the initial wavefront away from the source.  In empty space, this is a sphere of radius tψ.


Ia. Point source (wave), Delay Only

If disturbances propagate in the air ideally (as a wave), μ reaches an arbitrary point x after a delay of T seconds.  Considering only delay, the signal at x is identical to μ at some previous time t−T:
       
where T is given by the distance from x to m:
       
If S is a point source, X(t, μ, θ) is identical in all directions (we can ignore θ and φ),
       
and there is a single generating function (σ is everywhere zero):


Now, consider the microphones at L and R.
If N is the number of samples corresponding to delay T over distance ||x − m||,
   
the recordings v_L and v_R can be given as delayed copies of μ:

where
     
The delay N_D between L and R is given by
     
N_D is positive if ||b|| > ||a||  (...if the object is closer to the Right Microphone).

Ib. Attenuation by distance

The attenuation is a function of the distance the signal has traveled.  Time (delay) and distance vary proportionately, and the variables can be exchanged accordingly.  These transformations should be flexible. Use a placeholder, A(T):


X(t) is unchanged.

The signal is attenuated over the total distance traveled, and not the straight-line distance from source to microphone.  In empty space, these two distances are the same.  They will differ for reflections, transmission through other materials, moving objects, changes in the medium such as wind, etc.

Notes:

As the signal travels outward, the wavefront lies on the surface of a sphere of increasing radius.
Surface area of a sphere: 4πr²
The ratio of surface area of two circles with radius r₁, r₂ is  r₁² / r₂²
An identical quantity distributed evenly over the two surfaces are in the proportion r₂² / r₁²  units/m².
Let the reference distance be a sphere of radius 1.  Then at distance r,
     (SA of r)/(SA of reference) = 1 / r²

The real amplitude C of harmonic motion described by a pair of complex roots is the square root of the sum of the square of two complex-valued coefficients.
C = 1/(√(c₁² + c₂²)
Relevant functions:  Inverse-square: A(x) = f(k 1/x²),  Gauss:  A(x) = f(ke^-x2)

I will work out functions when I have a testable model.    The ears will decide. Choice+constraints breed efficiency. But as with light, the choices should be convincing approximations of real phenomena.  

Ic. Attenuation by angle

Suppose the Source is not a uniform, vibrating sphere. (It is not a point source. There is no such thing anyway.)  It is an oriented object.  S has both a position in space and an orienting unit vector e_s:   .   I have restricted the problem to two-dimensional space.  Generalization to 3D introduces independent complications.

Generating motion μ(t) propagates non-uniformly in space.  We can write

 where μ(t) is attenuated by an angular damping function  r = ξ(θ).

Equivalently,
     Suppose there is some phenomenon in the neighborhood of S, damping σ(t) in such a way that attenuation is a function of angle, or
     The function ξ(θ) is the volume envelope of σ(t).
The significance of the second statement will become apparent as the model develops.

For convenience, I will choose  e_S as the direction of maximum amplitude of X(t, θ).

• Example:   r = ξ(θ) is a cardioid with maximum value of 1, which occurs when θ = 0.  The e_S gives the orientation of the cardioid. Immediately, we have the magnitude of ξ(θ) in the direction of arbitrary vector x, for arbitrary orientation e_s
            
  Of particular interest to me are the equations
            
  ...with the natural constraints  -1 ≤ r ≤ 1, and/or where the area enclosed by the figure is constant as a, b, k are varied.
  {I want visual tools for manipulating the image of linked stereo files}
  {The mathematical elegance of the classic M/S stereo image warrant the approach.}    

The equivalence    is independent of the choice of function.
I accuse the family of cardioids of being not only fundamental to microphone design, but of generalizing naturally to spherical harmonics.    (The latter's a guess, but come on).

We have
     
I have declared the orientation to be a variable: e_s = e_s(t).  The signal arriving at arbitrary, point x is now
     
where the vector  = (m−x).

The principal difficulty with this representation is that we must know the values of e_s going back to t−T.  These difficulties will escalate as the model becomes more complex.  Formally, I will rewrtie the equations as convolution sums and integrals to manage the indices.  The fundamental analytic obstacle is to construct a transformation from listener to source.  The purpose of a formal approach is to minimize such difficulties at the time of playback.  Formal structures do not solve problems.  I will not let abstraction lead me around by the nose.
An infinite number of such approaches are possible.  I invent one.

We now have
       
Note that we only need to rotate either L and R about S, or the Source via e_s.  I choose to rotate e_s, and leave L and R fixed.  The variable    is replaced by normalized constants e_a and e_b, the unit vectors in directions   and    respectively.

II. The Signal is Composite/Convolved
Suppose the second signal,
     
 traverses m, and is subject to angular damping function
     

• Example:  , a pair of overlapping, circles.

M defines a source.  When the pressure is positive in one direction, it is positive in all.  How can we use this to construct an image?

Suppose the vibrating object is [can be written as] the convolution of two distinct functions.  Declare  an arbitrary point m.  Adopt X(t) as above: μ(t) propagates identically in all directions from this point; attenuated by an envelope.
Now consider the second motion, σ(t).   If the origin of σ is not be perfectly coincident with m, the disturbance σ(t) will traverse m in a uniform direction.   In fact, every motion not coincident with m will traverse it a distinct direction.

This is exactly what we need to distinguish an image in the neighborhood of a point.
{Side signal mirrored across x-axis, but opposite "sign".  σ(t) added to μ(t) on one side of sigma implies subtraction on the opposite side.  σ(t), uncorrelated to μ(t) is a line gradient, not an image.  M/S imaging immediately offers a means to ask a fundamental question:
Proposition: to widen the locus of sources about the point m, in such a way that a coherent image is constructed from two uncorrelated mono files.}

Case I: M and S are not correlated; i.e. the locus of points 1/2(1+cosθ) does not produce, when added (or convolved according to some function) to S, a field.  The source is two uncorrelated mono files.

Case II: M and S are correlated.  That is, it is also true that the signal at m relates all points not-coincident-with-m.  Then the source is an image of arbitrary spatial resolution.

Proposition: To determine conditions necessary and/or sufficient to ensure M and S are an image
Proposition: To determine/define the cases under which it is desirable to
Proposition: To determine conditions necessary and/or sufficient to add (convolve) arbitrary signals in such a way that  the relationship between the distinct objects is created in the image L R, heard at O'.  (Example:  Make sure all sound sources have directional noise components, and when mixing different sources correlate the noise.)
Proposition: To divide up the spatialization/audio pipeline into operational components:
   1) The bulk of imagery is defined at the beginning and end of the pipeline by dense stereo image sources, and precomputed perspective transformations immediately prior to sample playback.  Identical to lighting: the bulk of the information for transformation is a) precaptured, compact, variation-dense images   and b) transforming those images only when they are seen by a view vector.  Light is not actually ferried from place to place: images add to themselves based on the relative locations of light sources and viewer.   Images are themselves sources of light (diffuse channel+global illumination = lit scene;  image textures are of course captured from objects which are lit).  We are generally concerned with ferrying large sources of white light from place to place.
 2) Define sources of noise as a principal element of spatial construction, transformation, and coherence, by interaction with the environment and audio sources.
 3) ddefine global (audio) illumination.
Background noise with no location.  Follow the model of lighting.  Precompute transformations with the environment.  Mono nose file, looped, panned center provides no spatial information. Instead:  A primary purpose of background noise should be to glue together the sounds into a coherent environment.  Bad: Base global illumination lights objects by 1)adding low levels of broadband (noisy) light according to surrounding geometry/occlusion.  2) turning up and modulating pre-lit images defined on materials, 3) baking these values to tables/textures 4) adding these according to viewer position and angle.
 4) Define methods for converting point sources (mono files) to spherical images.  If the inside of the sphere is the operational space of our transformations, what can be done with mono files to achieve rich spatial properties  at the surface of the sphere?  Focus on widely applicable phenomena and types of motion: electricity, displacement, sympathetic vibration, irregularities of shape, buzzes, friction, strikes, transmission through materials, etc.

 

The Fourier Transfom - Periodicity II

THE SUM OF TWO PERIODIC FUNCTIONS

Given two elementary periodic functions,
          

When is the sum    periodic?  If it is periodic, what is the period?

1.  WHEN IS THE SUM PERIODIC?

f_i(t) is periodic with period T_i= 1/c_i
f_j(t) is periodic with period T_j = 1/c_j

If the sum is periodic then there exist two integers k₁, k₂ such that
          
That is, some multiple of T_i is also a multiple of T_j.   Take two tires, place them next to each other.  Mark a vertical lines down the side of each tire. Now spin them at two different speeds, 2πc_i and 2πc_j.  If those two lines are ever vertical at the same time in the future, then each wheel has at that moment completed a revolution.  For the tire f_i this is the k₁^th revolution.  For the the tire f_j this is the k₂^th revolution.

If no such integers exist, the sum is not periodic.  Putting the k's on one side:
        

So we have a simple rule:
If T_i/T_j is irrational, then there are no such k₁, k₂ and y(t) is not periodic.

2-1. DETERMINE THE PERIOD

If we can find on pair k₁, k₂ then we can find an infinite number of them:  multiply both sides by some integer n:
       
nk₁, nk₂ are integers which satisfy our query. To determine the fundamental period of y(t), we seek the least integers that satisfy k₁, k₂.  I'll do this by reducing fractions as I work. In short,

■ Procedure:   Given 
          
   1. Reduce m/r and n/s to lowest terms:    M/R,  N/S.
   2. Set c₁= the greatest common factor of the numerators M and N.
           c₂ = the greatest common factor of the denominators R and S.
 Then y(t) is periodic with period
          

2-2. THE DETAILS:

Introduce positive integers m, n, r, s;
Rational numbers P = m/r,      Q = n/s;
Distinct irrational numbers φ, θ.   (with all rational factors removed)

Reduce m/r and n/s (remove common factors, if any), and let
     
Expressed as fractions in lowest terms,
     
Introduce elementary periodic functions f_i(t), and corresponding periods T_i:
       

       

       

Determine  the period of y(t), for the following periods of the f(t). 
(a)  Integer periods. 

Let   .
The fundamental period is given by the least integers k₁, k₂ such that:
       
We have
       
Remove common factors of m and n:
       
Using T = k₁T₁ . . . y(t) is periodic with period
          

(b) Rational periods.  

Let  .  
If y(t) is periodic, there are integers k₁ and k₂ such that   T = k₁T₃ = k₂T₄.
       
To put k₁/k₂ in lowest terms, reduce S/R and M/N. (S/N and M/R are already reduced).
       
Using T = k₁T₃,  . . . y(t) is periodic with period
       

(c) One signal irrational.  

Let  .
We know that the ratio of periods cannot be rational.  y(t) is not periodic.
Specifically, we must have
       
But φ didn't stop being irrational. There are no such k₁, k₂.

(d) Both signals irrational

Let  .
If φ and θ were multiples of the same common irrational number, then the sum is still periodic.  (In the examples above, all coefficients of t have the common factors 2 and π.  This does not affect whether or not their sums are periodic.)

 θ and φ are distinct rational numbers;  f₅(t) + f₆(t) is not periodic.

■Next Question: At what time t_m does the maximum value of y(t) occur?

The Fourier Transform: Periodicity

{Diagrams Forthcoming}

DEFINITION:
Periodicity:  f(t) is periodic with period T if  there is some constant T such that
     f(t) = f(t + nT),    for every integer n, for all t.

EXAMPLE:
    Let y(t) = sin(t).
    Is y(t) periodic?  If so, what is the period?

I find a geometric solution the most straightforward.  Draw a circle with radius  r = 1.  Let O be the center.  For convenience, let O also be the origin of an x-y coordinate space.

Make a space ship.  Attach it to the circle where it intersects the positive x-axis: (1,0).    This is my starting position.  I have not moved yet.    Draw an arrow showing the direction the ship will travel around the circle.  I have chosen to go counter-clockwise.

My ship needs a destination.  Inscribe a right triangle in the circle, with central angle θ; hypotenuse r.    The legs of the triangle are then cos(θ), sin(θ).  It's up to me what I want the range of θ to be, around the circle.  I could divide the circle up into 360 evenly spaced, radial increments. That would be degrees, and super inconvenient right now.

So I'll do this.  From my starting position, trace the circle counter-clockwise to the vertex of the inscribed triangle.  Suppose I can measure that arc length, s, as a fraction of the whole circle.  I'll use that number for θ, too:
     Let θ = s, the arc length.  This is radians.
I want to make this as easy as possible, so I'll travel at a constant speed of 1 radian/sec.   After 1 second, my angle measure and distance along the circle are also 1:     .

The ship has an (x, y) position.  Call this position r.   Right now that position  is   r = (1,0).   As functions of the angle θ  from our starting position, the coordinates are:    x(θ) = cos(θ),    y(θ) = sin(θ).   But I've decided to let θ = s. Hence, the position r is given by:
       
Now I need a way to measure arc length along a circle.

Hahahaha just kidding.  The circle has a diameter, D=2.  To get from (1,0) to (-1,0), in a straight line (along the diameter), the distance is 2.   But for the space ship traveling along the circle, the distance from (1,0) to (-1,0) opens two hemispheres: one whole circle.  If we don't know the distance around the circle, we should assign it a letter.   For a Diameter of 1, I'll call the length around the whole circle π.   ... Right?

That is, the ratio of the Circumference to the Diameter of a circle  is  π.  Our circle has a diameter of 2, and so its circumference is 2π.  From any point on the circle, if you travel this distance in a single direction, the whole circle is traversed, and you are back where you started.   You may go around as many times as you want.  In other words,
       
But I made s = t.  So,
       

That is, 

The functions

are periodic with period
 

In radians, as long as the circle has a radius of 1, angle measure always equals arc length.  But time doesn't.  I can change the relationship to time by changing speeds:

■ Scalar multiplication of t:     y(t) = sin(ct) ,   c an arbitrary constant.
Now we are traveling at a constant speed c around the circle.  Now our position at time t is
       
And     Distance = Rate × Time:
       
Note that the position vector is still r(s) = (cos(s), sin(s)).

Calculate the Period: 
If our current position is s₀ and the time is t₀ how long does it take to circle once and get back to s₀?
      
This calculation is independent of the values of t₀ and s₀.  Both the circumference and our speed are constant.  If complete n circles, our final position is still s₀.   As above, this holds no matter where we are on the circle:    .   However, time will march onward:
       
So what time is it when we finish our first lap, anyway?
       
Time to check myself.  Using ordinary substitution, I should get r(t) = r(t + 2π/c).

Good.

I'd like to talk about speed, and trips around the circle, without writing π everywhere.  Let's agree right now to measure our speed as some multiple of 2π:

■ Speed normalized to the circle:   y(t) = sin(2πct)
2πc is still just a constant.  The problem is the same.
       
Calculate the Period: 
How long does it take to complete one lap?  Distance = Rate × Time:
      
And what time is it when we complete one trip around the circle?
       
Much better.  Our speed is  c circles/second, and it takes 1/c  seconds to complete one circle.  This is not a magical property of numbers, I've done it on purpose to describe the problem.
How can my little spaceship problem be useful?  Change the word circles to cycles, and this is the Hertz scale.  But for the spaceship to follow a signal front around, I  have a lot more work to do.  Onward.

■ Time offset :    y(t) = sin(2πct + δ)
δ has no effect on the relationship between t and s.
The period  is also unaffected by δ.  s + 2πn = s holds for all s.    It holds for s + δ.  Thus,

The functions

 are periodic with period

■ Additional Considerations
Say we are both at (1,0).  I am trapped in my spaceship; you may walk freely about.  If you and I want to meet at (-1,0),  you have one, fastest way to get there: walk straight left along the x-axis. For me there are two, equal paths, through positive y and  negative y.   When we both stand at (-1, 0), for the return trip I again have two choices.  To perform a round-trip, you have one possible course and I have four.

Now suppose you stand at (1,0), and I zoom by.  If you only measure from my current position, I may be traveling in any of the infinite number of other unit circles which pass through the point (1,0).  By intersecting my circle at two points (in a secant, such as a diameter) you can determine which of these courses I traverse.  Without this information, you cannot accurately assess the distance between us and direction at any point in time.  Your distance calculations will be fine if you stand still, but once you begin to move about, both the distance and direction are unknown.  The same information can be derived (! differentiation) from my curve of motion.  Observing my spaceship in the near vicinity of (1,0) should suffice.

■ Next Question:  When is the sum of two periodic functions periodic?