% $Header: /home/rdv/quantum/genetic-paper/RCS/eaqc-paper.tex,v 1.12 2005/05/29 21:34:00 rdv Exp rdv $
% now how do I set the paper size?
% \documentclass[twocolumn]{article}

\documentclass[onecolumn,doublespace]{article}
\usepackage{times}
\usepackage{graphics}
\usepackage{lscape}
\usepackage{subfigure}
\usepackage{graphicx}
\usepackage{color}
\usepackage{amsmath,amsfonts,amssymb}
\usepackage{kbordermatrix,blkarray}
\pagestyle{plain}

\bibliographystyle{unsrt}   %>>>> makes bibtex use spiebib.bst

% \input{psfig}
\input{epsf}
% \epsfverbosetrue

%\def\comment#1{$\Rightarrow${\small\em #1}}
\def\comment#1{}
\newenvironment{chapterquote}
  {\begin{quotation}
      \sf}
  {\end{quotation}}
\def\cq#1#2{\begin{chapterquote}#1\flushright\textbf{#2}\end{chapterquote}}


\newcommand{\icol}[1]{% inline column vector
  \left(\begin{smallmatrix}#1\end{smallmatrix}\right)%
}

\newcommand{\irow}[1]{% inline row vector
  \begin{smallmatrix}(#1)\end{smallmatrix}%
}

\begin{document}
\title{Optimization Theory (DS2) Lecture \#2\\
Examples of Linear, Integer and Graph Problems
}
\maketitle
\begin{abstract}
  Today we are primarily going to look at how to \emph{formulate}
  problems, starting from word descriptions of the problem, into a
  rigorous mathematical form.  How we \emph{solve} those problems will
  come during the course of the semester.
\end{abstract}

\emph{Text and examples adapted from \emph{A Gentle Introduction to
    Optimization}.}

\section{Introduction}

Today we are going to run through a set of examples.  Ideally, you
should follow up on this by reading the corresponding sections in the
book and trying some of the exercises.

Examples:

\begin{enumerate}
\item Linear Program: WaterTech (Sec. 1.1 from the textbook)
\item Integer Program: KitchTech, the knapsack problem (Sec. 1.3.2)
\item Graphs: shortest path problem (Sec. 1.4.1, 1.5.2)
\item Graphs: minimum cost perfect matching (Sec. 1.4.2, 1.5.1)
\end{enumerate}

\section{Linear Program: WaterTech}

WaterTech makes four types of products, each of which requires some
amount of time on Machine 1 (say, a milling machine) and a certain
amount of time on Machine 2 (say, a polishing machine).  It takes both
skilled and unskilled labor to make the products, and of course each
sells for a certain price.  How can we maximize profit?

Table of times and selling price:

% \begin{table}
\begin{tabular}{cccccc}
Product & Machine 1 & Machine 2 & Skilled labor & Unskilled labor & Unit sale price \\ \hline
1 & 11 & 4 & 8 & 7 & \$300 \\
2 & 7 & 6 & 5 & 8 & \$260 \\
3 & 6 & 5 & 5 & 7 & \$220 \\
4 & 5 & 4 & 6 & 4 & \$180 \\
\hline
\end{tabular}
% \end{table}

We have the following constraints:

\begin{enumerate}
\item 700 hours of time are available on Machine 1.
\item 500 hours of time are available on Machine 2.
\item Up to 600 hours of skilled labor at \$8/hour can be purchased.
\item Up to 650 hours of unskilled labor at \$6/hour can be purchased.
\end{enumerate}

{\bf Variables.} Our product ``names'' are the numbers 1 to 4, so
let's call $x_i$ the number of each product we decide to make, for $i
\in \{1,2,3,4\}$.  (Later we will see that sometimes you have to add
artificial variables, but in this case we'll stick with the basics.)

{\bf Constraints.}  The total amount of time used on Machine 1 has to
be less than 700 hours, and can be written
\begin{equation}
11x_1 + 7x_2 + 6x_3 + 5x_4 \le 700.
\end{equation}
Likewise, the constraint on Machine 2 is
\begin{equation}
4x_1 + 6x_2 + 5x_3 + 4x_4 \le 500.
\end{equation}
You can see these equations by going down the columns labeled Machine
1 and Machine 2 in the table above.

We don't yet know how much labor we're buying, so let's call unskilled
labor $y_u$ and skilled labor $y_s$.  Going down the corresponding
columns, we get the constraints
\begin{align}
8x_1 + 5x_2 + 5x_3 + 6x_4 & \le y_s \\
7x_1 + 8x_2 + 7x_3 + 4x_4 & \le y_u
\end{align}

{\bf Objective function.}  This is the thing we're optimizing for:
profit!  Let's call our income from sales $I$, costs $C$, and profit
$P$.  Obviously,
\begin{equation}
P = I - C
\end{equation}
and we want that to be positive and as big as possible!  We can read
the income off the right hand column of the table,
\begin{equation}
I = 300x_1 + 260x_2 + 220x_3 + 180x_4
\end{equation}
and the labor costs are
\begin{equation}
C = 8y_s + 6y_u.
\end{equation}

\subsection{Formulation}

Finally, this gives us the formal formulation of the formulas for this
problem:

Maximize
\begin{equation}
300x_1 + 260x_2 + 220x_3 + 180x_4 - 8y_s - 6y_u.
\end{equation}
subject to the constraints
\begin{align}
11x_1 + 7x_2 + 6x_3 + 5x_4 & \le 700 \\
4x_1 + 6x_2 + 5x_3 + 4x_4 & \le 500 \\
8x_1 + 5x_2 + 5x_3 + 6x_4 & \le y_s \\
7x_1 + 8x_2 + 7x_3 + 4x_4 & \le y_u \\
y_s & \le 600 \\
y_u & \le 650 \\
x_1,x_2,x_3,x_4,y_s,y_u & \ge 0.
\end{align}

The first four of those constraints are the equations we derived
above, but don't forget the bottom three -- without them our
techniques can easily yield an unphysical solution such as buying a
\emph{negative} amount of labor or building an infinite number of some
product!

\section{Knapsack Problem}

(Our first real integer programming problem!  Might not get there
today, but the problem is from 1.3.2.)

\section{Shortest Path Problem}

A \emph{graph} is a set of vertices $V$ and a set of edges $E$, where
each edge connects to two of the vertices, or nodes.  For some
problems, we may begin bg guaranteeing that the whole thing is
connected, for others, we may not.

You will see this problem in several forms in different fields; the
way networking people approach it is somewhat different from the way
we will see it here.  Today, we will formulate it as an integer
problem.  (Later in the semester, if we have time, we will see another
means of solving this problem, called \emph{shortest path first}, by a
dude named Edsger Dijkstra.)

See Fig. 1.9 in the text.  We have four vertices, $s$ (our source),
$t$ (our target), $a$ and $b$.  Let's first see the \emph{adjacency
  matrix}, which contains 1 if two vertices are connected by an edge,
and 0 if not (and along the diagonal):
\begin{equation}
\bordermatrix{
& s & a & b & t \cr
s & 0 & 1 & 1 & 0 \cr
a & 1 & 0 & 1 & 1 \cr
b & 1 & 1 & 0 & 1 \cr
t & 0 & 1 & 1 & 0
}
\end{equation}
Obviously, this is an $n \times n$ matrix for a graph with $n$ nodes.
If the edges are \emph{weighted}, e.g. representing distances between
two points, we have the \emph{weighted adjacency matrix} (or sometimes
the \emph{distance matrix}):
\begin{equation}
\bordermatrix{
& s & a & b & t \cr
s & 0 & 3 & 4 & \infty \cr
a & 3 & 0 & 1 & 2 \cr
b & 4 & 1 & 0 & 2 \cr
t & \infty & 2 & 2 & 0
}
\end{equation}

The numbers are known as the \emph{cost} or \emph{weight}.  (Often
edges in a graph have a direction, known as a \emph{directed graph} or
\emph{digraph}, but here we are dealing only with undirected edges.)

\subsection{Paths}

A \emph{path} is a list of edges that connects $s$ to $t$.  Their
order matters, and the tail of each edge has to be the head of the
next edge.  No node can be repeated.  So for this simple graph, we
have four possible paths:
\begin{align}
sa, at \\
sa, ab, bt \\
sb, bt \\
sb, ba, at.
\end{align}

Each edge has a cost, as above.  If we establish the order of our
edges to be $sa, sb, ab, at, bt$, then we can write the vector
$\vec{x} = (1, 0, 0, 1, 0)^\intercal$ to represent the path $sa, at$.
(Question: What is the length of this vector?)

If we write the edge costs as a vector, also, then we can calculate
the \emph{path cost} as
\begin{equation}
C = (3, 4, 1, 2, 2)\vec{x}
\end{equation}
which is the same as
\begin{equation}
C = \sum_{e \in P}c_e = \sum_{e \in E}c_e x_e
\end{equation}
where $c_e$ is the cost for the edge $e$, $P$ is the path, and $E$ is
the (complete) set of edges.

\subsection{Cuts (or Partitions)}

How many different ways can we split our graph into two parts, with
$s$ in one part and $t$ in the other?  If there are $n$ total nodes in
our graph, there are $2^{n-2}$ ways (because we start with the
assumption that $s$ is in one half and $t$ in the other).  In our
example, we only have two other nodes, $a$ and $b$, so we have $2^2 =
4$ possible sets.  We'll refer to $\delta(U)$ as the \emph{cut} or
\emph{st-cut} of the set $U$, and the four possibilities are $s$ plus:
$\{\},\{a\},\{b\},\{a,b\}$.  Then our possible cuts are:
\begin{align}
\delta(\{s\}) & = \{sa, sb\} \\
\delta(\{s,a\}) & = \{at,ab, sb\} \\
\delta(\{s,b\}) & = \{sa, ab, bt\} \\
\delta(\{s,a,b\}) & = \{at, bt\}.
\end{align}

It turns out that a path that reaches from $s$ to $t$ has at least one
edge in \emph{every one} of those cuts above.

\subsection{Formulation}

{\bf Variables.} As above, $x_e$ is 0/1 for each edge $e$.

{\bf Constraints.} Our strategy for the integer problem formulation of
this is to have a constraint for each one of those sets $U$ above,
saying that we have at least one edge from each of them.  (Question:
how many constraints is this?  Recall that we said last time that the
number of constraints must be finite, but we didn't say it has to be
small!)

{\bf Objective function.}  Our cost $C$, above.

We can write this down more mathematically as:

Minimize $(3, 4, 1, 2, 2)\vec{x}$ subject to
\begin{equation}
\bordermatrix{
& sa & sb & ab & at & bt \cr
\{s\} & 1 & 1 & 0 & 0 & 0 \cr
\{s, a\} & 0 & 1 & 1 & 1 & 0 \cr
\{s, b\} & 1 & 0 & 1 & 0 & 1 \cr
\{s, a, b\} & 0 & 0 & 0 & 1 & 1
}\vec{x} \ge \vec{1}
\end{equation}
and
\begin{equation}
x_e \ge 0, x_e \in \mathbb{Z}
\end{equation}
where I've used $\vec{1}$ to mean the vector of all ones of the
appropriate length, and $\mathbb{Z}$ is the set of all integers, that
is, every $x_e$ has to be an integer.  This big matrix constraint
means that every element of the vector resulting from multiplying this
matrix times our solution vector $\vec{x}$ has be at least one.

Note that many non-paths are technically solutions to this problem,
but it can be proven that all \emph{optimal} solutions will be
st-paths.


\section{Minimum Cost Perfect Matching}

(Might not get there today, but the problem is from 1.4.2 and 1.5.1.)

\end{document}
