\documentclass[11pt]{article}
%\usepackage{psbox}
\usepackage{amsmath,amssymb,complexity}

\textwidth=6in
\oddsidemargin=0.25in
\evensidemargin=0.25in
\topmargin=-0.1in
\footskip=0.8in
\parindent=0.0cm
\parskip=0.3cm
\textheight=8.00in
\setcounter{tocdepth} {3}
\setcounter{secnumdepth} {2}
\sloppy


\newtheorem{example}{Example}
\newcommand{\ntime}{\hbox{NTIME}}
\newcommand{\nspace}{\hbox{NSPACE}}
\newcommand{\conspace}{\hbox{co-NSPACE}}
\newcommand{\pspace}{\hbox{PSPACE}}
\newcommand{\lspace}{\hbox{L}}
\newcommand{\conp}{\hbox{coNP}}
\newcommand{\exptime}{\hbox{EXPTIME}}
\newcommand{\elem}{\hbox{E}}
\newcommand{\nl}{\hbox{NL}}
\newcommand{\bpp}{\hbox{BPP}}
% self added command
\newcommand{\pp}{\mathrm{P}}
\newcommand{\ph}{\mathrm{PH}}
\newcommand{\sigpi}{\mathrm{\Sigma_2\cap \Pi_2}}
\newcommand{\ppoly}{\mathrm{P/poly}}
\newcommand{\np}{\mathrm{NP}}
\newcommand{\sat}{\mathrm{SAT}}
\newcommand{\eps}{\varepsilon}
\newcommand{\boolf}{\mathbb{B}^n}

\newcommand{\nregexp}{\hbox{NREGEXP}}

\newcommand{\tqbf}{\hbox{TQBF}}
\newcommand{\threesat}{\hbox{3SAT}}
\newcommand{\cvp}{\hbox{CVP}}
\newcommand{\stconn}{\hbox{STCONN}}
\newcommand{\ispath}{\hbox{ISPATH}}


%\newcommand{\class}{\hbox{$\mathbb{C}$}}
%\newcommand{\class}{\hbox{$\mathbf{C}$}}


\newcommand{\lep}{\leq _{\hbox{P}}}
\newcommand{\lel}{\leq _{\hbox{L}}}
\newcommand{\aspace}[1]{{\rm ASPACE}(#1)}
\newcommand{\atime}[1]{{\rm ATIME}(#1)}
\newcommand{\spa}[1]{{\rm SPACE}(#1)}
\newcommand{\ti}[1]{{\rm TIME}(#1)}
\newcommand{\ap}{{\rm AP}}
\newcommand{\al}{{\rm AL}}

\begin{document}
\input{preamble.tex}
\lecture{10}{Apr 13, 2009}{Jayalal Sarma M.N.}{Youming Qiao}

In the last few lectures, we saw the circuit model of computation,
various resources associated with them, and how some of them compare
with the complexity classes defined based on Turing machines. In this
lecture we will to give an introduction to a ``holy grail'' of
complexity theory; namely proving circuit lower bounds. 

\section{Explicit Lower Bound question}

With the famous $\P$ vs $\NP$ question in mind, the aim is to prove
lower bounds for circuit parameters (like size, depth etc) required
for any {\em uniform} circuit family to compute specific Boolean
functions (say in $\NP$). For example, if we prove a super-polynomial
size lower bound\footnote{We call this a lower bound against
  polynomial size Boolean circuit families.} for any {\em uniform}
circuit family computing a specific Boolean functions (say in $\NP$)
then we separate $\P$ from $\NP$.

An even stronger aim (hence may be difficult to achieve) can be is to
relax the constraint of uniformity. Let us say that we want to argue
that for an explicit Boolean function $f$ in $\NP$ there does not
exist a poly size circuit family at all (so no uniformity machine,
no-matter how powerful it is, cannot find it anyway). This relaxation
is done more because, we do not know how to effectively exploit the
{\em uniformity} constraint even if it is included.

The circuit lower bounds against non-uniform circuits does imply lower
bounds against uniform circuit families as well. But we will relax
this question further. Do we know if there exist Boolean functions
that requires super-polynomial size circuits? Yes, and this is an
important result which goes back to Shannon (1948). We will see this
counting argument today in some detail. In fact it indicates that {\em
  most of} Boolean functions defined actually require super-polynomial
size circuits. Now, can we find some specific ones? This leads to the
explicit Lower bound question.

\begin{definition}[Explcit Lower Bound Question]
Is there an explicit Boolean function\footnote{preferably one in
  $\NP$, but in this case the notion of explicitness is weaker,
  because if ``exists'' a function in $\NP$ that requires super-poly
  size circuits, then so does the $\SAT$ function, which is explicit.}
$f:\{0,1\}^n \to \{0,1\}$, for which any circuit family $\{C_n\}_{n
  \ge 0}$ computing it must have super-polynomial size?
\end{definition}

As we argued, this is stronger than $\P$ vs. $\NP$ question which is
is not answered yet. But we believe that this formalism is more useful
for the techniques that we have and that provides the motivation to
study the circuit lower bound question. However, despite lot of
efforts by the best minds, we do not have strong reasons to believe
that we are close to proving it or disproving it.

There has been a lot of progress if we restrict the circuit family
that we consider. This line of research was very successful in the
1980's, and several distinguished results, including {\sc PARITY}
$\notin \AC^0$ were proved. We will see several of these results in
the next few lectures.

The specific agenda for today is to present (1) a conditional answer
to the explicit lower-bound question (2) an unconditional answer to
the lower-bound question, but in the non-explicit way.

\section{Conditional Lowerbound (Karp-Lipton-Sipser Theorem)}

Karp, Lipton and Sipser showed that $\NP$ is unlikely to have
polynomial sized circuit, by showing that a collapse of the $\PH$ is
implied by it. We believe that this is not the case. A way to view
this is as s conditional lower bound, where the condition is widely
believed to be satisfied. We will present this result in this section.

Before going into the main theorem, we will review the three
aspects of a language being computed by polynomial sized circuit.

\subsection{Three Descriptions of Polynomial Circuits}

Some notations: for a language $L\in \{0,1\}^*$ denote $L_n=\{x \in L
: |x|=n\}$. And $\N$ is denoted as the set of natural numbers.

\begin{definition}[Polynomial circuits] 
A language $L$ is said to be computable by a polynomial circuit family
if there exists a family of circuits $\{C_i\}_i\in \N$ such that
$|C_n| \leq p(n)$ for some polynomial $p(n)$ and $C$ computes $L$ :
that is, $\forall x: x \in L \iff C_{|x|}(x) = 1$.
\end{definition}

\begin{definition}[Sparse sets] 
A set $S\subset \{0,1\}^*$ is called a sparse set if there exists a
polynomial $p(n)$ such that $|S_n|\leq p(n)$.
\end{definition}

\begin{definition}[P-time TMs with $\poly$ advice] A language
$L\in \ppoly$ if there exist $L'\in \pp$ and a polynomial bounded
  function $l: \N \rightarrow \{0, 1\}^M$, $|l(n)|\leq p(n)$ for some
  polynomial $p(n)$, such that:
\begin{eqnarray*}
x\in L \iff \langle x, l(|x|)\rangle \in L'
\end{eqnarray*}
and $l$ is called the advice function.
\end{definition}

\begin{proposition} \label{prop:equivalent}
For a language $L\in \{0,1\}^*$, the following three propositions
are equivalent:
\begin{enumerate}
\item $L$ has polynomial circuits $\{C_n\}_{n\in N}$;
\item $L \in \ppoly$;
\item $L \in \pp^S$ for some sparse set $S$.
\end{enumerate}
\end{proposition}

\begin{proof}
We will prove (1)$\iff$(2), and (1)$\iff$(3).

\paragraph{$(1)\Rightarrow (2)$} Given input $x$, the advice function just maps $n=|x|$ to
the description of $C_n$, and the Turing machine will simulate the
computation of $C_n$ on $x$.
\paragraph{$(2)\Rightarrow (1)$} We need to construct a circuit family $\{C_n\}_{n\in N}$ computing
$L$. To construct $C_n$ we can hardwire the result of $l(n)$ (we can
do so since we just need to show the existence of such a circuit
family) and then simulate the computation of the corresponding
polynomial Turing machine that accepts $\langle x, l(|x|)\rangle$.
(This can be done since we already know $\pp$ has polynomial
circuits, and $l(n)$ is polynomially bounded.)
\paragraph{$(3)\Rightarrow (1)$} For Turing machine $M$ computing $L\in \pp ^S$,
on input $x$ it would only query strings in $S$ the lengths of which
are polynomial of $n$. And the number of queries is bounded by some
polynomial, too. So given $L_n$, the total length of queries made by
$M$ for $L_n$ is a polynomial of $n$. Then we just need to hardwire
those queries into the circuit and the circuit would simulate the
computation of the Turing machine with oracle without difficulty.
\paragraph{$(2)\Rightarrow (3)$} We would turn the advice function $l(n)$ into a
sparse set $S$, such that the machine would get $l(n)$ from $S$ by
querying oracles.

The trick is to construct a set $S$ as follows (suppose $|l(n)|$ is
bounded by $p(n)$:

\begin{eqnarray*}
S=\{\langle1^{p(n)},p_n \rangle \mid p_n \textrm{ is a prefix of }
l(n), n\in N\}
\end{eqnarray*}

It is easy to see that $S$ is a sparse set. Then the machine would
operate as follows: given $x$ of length $n$, to get $l(n)$, it would
query $\langle1^{p(n)}, 0\rangle$ and $\langle1^{p(n)}, 1\rangle$ to
get the first bit of $l(n)$. Having found a prefix $p$ of $l(n)$,
the machine would query $\langle 1^{p(n)}, p0\rangle$ and $\langle
1^{p(n)}, p1\rangle$ to determine the next bit, until the machine
find that $l(n)$ is already fully extended.
\end{proof}

\begin{theorem}\label{thm:klp_theorem} {\em (Karp-Lipton-Sipser Theorem)} If $\np \subset
\ppoly$, then $\ph=\sigpi$.
\end{theorem}

Another way to state KLS theorem would give us the conditional
circuit lowerbound theorem, given Proposition \ref{prop:equivalent}.

\begin{corollary} {\em (Conditional circuit lowerbound)} If
$\ph \ne \sigpi$, then $\sat$ function requires super-polynomial size
  for any circuits computing it.
\end{corollary}

\begin{remark}
Note that in Proposition \ref{prop:equivalent}, the assumption is that
$L$ is Turing-reduced to a sparse set. A natural question is whether a
stronger assumption would imply a stronger consquence. For many-one
reductions, which is a stronger condition, Mahaney proved in 1982 that
a stronger consquence can be derived. {\bf Mahaney's Theorem:} {\em if
  $\sat$ can be Karp (many-one) reduced to a sparse set, then
  $\np=\pp$.} Ogihara and Watanabe (1991) strengthened Mahaney's
theorem as : if $\sat$ can be Turing reduced to a sparse set where the
reduction asks only constant number of queries to the sparse set, then
$\P = \NP$.  It is open if this is true for $O(\log n)$ queries.
\end{remark}

\subsection{Proof of KLS Collapse Theorem}

In this section we prove Theorem \ref{thm:klp_theorem}.

\begin{proof}
We will prove $\Pi_2\subset \Sigma_2$, which implies $\ph$ collapses
to $\sigpi$.

Starting with $L\in \Pi_2$, there exists a polynomial Turing machine
$T$ such that

\begin{eqnarray*}
x\in L \iff \forall y \exists z [T(x,y,z)=1]
\end{eqnarray*}

Then using Cook's theorem, which shows that $\sat$ is
$\np$-complete, the $\exists z [T(x,y,z)=1]$ part can be replaced
with satisfiable, polynomial-size CNF formula $\psi$ that takes
$(x,y)$ as input. So we have

\begin{eqnarray*}
x\in L \iff \forall y [\psi(x,y)\in \sat]
\end{eqnarray*}

Since $\psi(x,y)$ is of polynomial size, say $p(n)$, the assumption
$\np \subset \ppoly$ would give us a polynomial-size circuit $C$
that solves $\sat$ of size $p(n)$. So one can turn the above $\Pi_2$
computation into a $\Sigma_2$ computation by guessing $C$ first,
then use $C$ to solve the $\psi(x,y)\in \sat$ part. That is:

\begin{eqnarray*}
x\in L & \Rightarrow & \forall y [\psi(x,y)\in \sat]  \\
 & \Rightarrow & \exists C \forall y [C(\psi(x,y))=1]
\end{eqnarray*}

However, for the inverse direction, things are a bit different. At
this time we requires all circuits of size $|C|$ should reject the
unsatisfiable formulaes. That is:

\begin{eqnarray*}
x\notin L & \Rightarrow & \exists y [\psi(x,y)\notin \sat] \\
 & \Rightarrow & \forall C \exists y[C(\psi(x,y))=0]
\end{eqnarray*}

The good news is that, if some circuit $C$ claims $\psi$ to be
correct, we can construct another circuit $C'$ making use of $C$ to
find the satisfying assignment. This is called the
"self-reducibility" property of $\sat$.

So given $C$ that claims to solve $\sat$ of size $\leq p(n)$, $C'$
operate as follows: on input $\psi$, $C'$ feeds $C$ with $\psi$, and
if $C$ claims $\psi$ to be satisfiable, $C'$ would feed $C$ with
$\psi_1=\psi_{x_1=T}$ and with $\psi_2=\psi_{x_1=F}$. If $C$ claims
one of them to be satisfiable, $C'$ would set $\psi$ to be the
satisfiable one and recursively do the above procedure. In the end,
$C'$ would get some assignment $s$ and check whether $s$ makes
$\psi$ satisfiable. Only if this is true $C'$ would accept $\psi$,
otherwise $C'$ just rejects.

So the above procedure makes sure that those "bad" circuits would
not affect our computation. And to get $C'$ from $C$ is efficient.
To conclude we have:

\begin{eqnarray*}
x\in L & \iff & \exists C \forall y [C'(\psi(x,y))=1]
\end{eqnarray*}

And the proof is complete.
\end{proof}

\section{Unconditional Lowerbound (Shannon's Theorem)}

Some notations: $\mathbb{B}^n$ denotes $\{0,1\}^{\{0,1\}^2}$, the set
of all $n$-ary Boolean functions. For an integer $n\in N$, $[n]$
denotes the set $\{1,2,\dots,n\}$.  We will now prove Shannon's
Theorem.

\begin{theorem}\label{thm:shannon}
Let $\eps>0$.  The ratio of $\boolf$ that can be computed by circuits
over $\mathbb{B}^2$ with $(1-\eps)\frac{2^n}{n}$ gates approaches 0 as
$n\rightarrow \infty$.
\end{theorem}

\begin{proof}
For $f\in \boolf$, define $S(f)\doteq\min \{S(C)\mid C \textrm{ is a
circuit over } \mathbb{B}^2 \textrm{ computing } f\}$. For $q, n\in
N$ define $N_{q,n}\doteq |\{f\in \boolf \mid S(f)\leq q\}|$. So the
claim becomes for $q=(1-\eps)\frac{2^n}{n}$, the following holds:

\begin{eqnarray*}
\lim_{n\rightarrow \infty} \frac{N_{q,n}}{2^{2^n}} = 0
\end{eqnarray*}

Next we will use some combinatorial method to bound $N_{q,n}$. The
crucial point is to realize that a circuit $C$ of size $q$ can be
encoded by the following map:

\begin{eqnarray*}
\tau : [q] \rightarrow ([q]\times \{x_i\}_{i\in [n]})^2 \times [16]
\end{eqnarray*}

This is because every internal gate has two predecessors, which can
be other internal nodes or the input nodes. Also every internal gate
can be of 16 types since the circuit is over $\mathbb{B}^2$. Denote
$T_{q,n}$ as the set of all such maps and $|T_{q,n}|=(16\cdot
(n+q)^2)^q$.

However, two points needs some consideration: the first point is
that some maps of this form are not valid circuits. We can ignore
this point since we want to show an upper bound. The second point is
that some different maps may describe the same circuit. We deal this
in the following.

Let $\sigma\in S_q$ be a permutation on $q$ elements, and define
$\sigma \tau$ in the obvious way. (If $\tau(i)=(j,k,l)$, then
$\sigma\tau(i)=(\sigma(j), \sigma(k), l)$, by defining
$\sigma(j)=\sigma(j)$ when $j\in[q]$, and $\sigma(j)=j$ when $j\in
{x_i}_{i\in[n]}$.)

For a circuit $C$, in principle every gate $v$ can be viewed as a
function over the inputs. Denote this function as $C_v$. A circuit
$C$ is said to be reduced if for every pair of internal nodes $v$
and $u$, $C_v\neq C_u$. The reduced circuit is important since for
the map $\tau$ representing it, the only permutation $\sigma$ that
would cause $\sigma\tau=\tau$ is just the identity permutation. So
it contributes the most "replications" in $T_{q,n}$. In other words,
a circuit can not have more than $q!$ replicas, so we have

\begin{eqnarray*}
N_{q,n}\leq q\cdot \frac{T_{q,n}}{q!}
\end{eqnarray*}

The $q$ factor appears since for a reduced circuit, different gate
contributes a different function.

Next we just need to work on counting. Let $d=16e$ and $q\geq n$,
then by Stirling's formula we have:

\begin{eqnarray*}
N_{q,n} & \leq & q\cdot \frac{(16\cdot (n+q)^2)^q}{q!} \\
 & \leq & q\cdot d^q \cdot \frac{(n+q)^{2q}}{q^q} \\
 & \leq & q\cdot d^q \cdot \frac{4^q q^{2q}}{q^q} \\
 & = & q(4dq)^q \\
 & \leq & (4dq)^{q+1}
\end{eqnarray*}

Let $c=4d$. Next we plug in $q=(1-\eps)^\frac{2^n}{n}$, and for
sufficiently large $n$ ($n\geq c(1-\eps)$) we have:

\begin{eqnarray*}
N_{q,n} & \leq & (2^n)^{(1-\eps)2^n/(n+1)} \\
 & = & 2^{(1-\eps)2^n+n}
\end{eqnarray*}

So we get the result.
\end{proof}

In terms of circuit lower bound, we have:

\begin{corollary}
Given $\eps>0$, most Boolean functions need more than
$(1-\eps)2^n/n$-size circuits when $n$ approaches infinity. 
\end{corollary}

But as we remarked initially, this does not give us any explicit
Boolean function which requires this sized circuits computing it.
Now, given that Shannon's bound is actually less than the obvious
$O(2^n)$ size bound for any boolean function, a natural question is
whether the size $\frac{2^n}{n}$ is actually sufficient.  This leads
us to Lupanov's theorem.

\begin{theorem}
Every Boolean function can be computed by circuits with
$\frac{2^n}{n}+o(\frac{2^n}{n})$ gates over the basis
$\{0,1,\oplus,\land\}$.
\end{theorem}

We will skip the details of the proof of this theorem.  It uses a nice
representation of Boolean fuction as ``ring sum expansions''.
Interested readers are referred to \cite{Vollmer}.

\bibliographystyle{plain} \bibliography{lecture10}

\end{document}
