Well, in part we aimed too high: My wife
was brilliant, quite broadly -- yes,
Valedictorian, Summa Cum Laude, Woodrow
Wilson, PBK, piano, clarinet, voice,
prizes in cooking, sewing, raising
chickens. But her family had her try to
be perfect and to dedicate herself to
saving the world.
She wanted a Ph.D. in mathematical
sociology to do social engineering of
social change to save the world and,
thus, get her praise, acceptance, and
emotional and financial security. And
those concerns filled her plate.
Well, at the roots there some anxiety,
from nature and/or nurture, was involved,
then some perfectionism and some fear of
criticism from powerful, influential
people -- net, call it a special case of
social phobia. That brought stress
which can bring depression. That slows
the work and in her case caused more
stress and more depression then clinical
depression. She was in a clinical
depression the day she got her Ph.D. She
never recovered and, net, didn't make it.
Took me a while to reinvent and learn the
basics of clinical psychology to
understand what was going on and what to
do about it. I did learn but nearly
always too late.
In trying to have a stable job so that I
could take care of her, I took a job at
IBM's Watson lab, in what we called
artificial intelligence (AI) to do
monitoring and management of large server
farms and networks.
Then IBM got sick and the lab phone book
went from 4500 full time names down to
1000 or so. The guy who hired me, a big
star who was deliberately ignored by the
higher ups, left for greener pastures --
eventually ended up with a nice place in
Malibu.
IBM Watson Research was run by a clique
of people who stuck together and blocked
out everyone else. At one point it
appeared that the company HQ in Armonk
tried to correct that situation.
Instead of or better than the AI, I had
some ideas: Detecting a problem in a
server farm or network is necessarily
essentially a statistical hypothesis test
with Type I (false alarm) and Type II
(missed detection) errors. So, want
hypothesis tests that keep down the
rates of the errors.
Since there's a lot of data readily
available and want to exploit it to keep
down the rates of the errors, also want
multi-variate inputs -- nearly all
hypothesis tests have only univariate
inputs. Also for such multi-variate data
can't hope to know any of the probability
distributions so need tests that are
distribution-free.
I don't think there were any such tests,
so I dreamed up some. I used some group
theory, summed used the classic S. Ulam,
guy on the left in
result LeCam called tightness (see P.
Billingsley, Convergence of Probability
Measures), and was able to permit
selecting false alarm rate in advance and
then getting that rate exactly. Asking
for all of the Neyman-Pearson result would
take more data than we had any chance of
having, but there was a somewhat useful
sense in which my work gave
asymptotically, for any selected false
alarm rate, the highest possible detection
rate with that false alarm rate.
I cooked up some synthetic data that was
challenging -- the critical region was
something like the red squares on a
checkerboard in several dimensions. My
work did fine.
I got some data from a cluster of
computers at Allstate, wrote some
prototype software, and confirmed the
false alarm rate empirically. And I
cooked up an algorithm to make the
computations nicely fast (in part I used
k-D trees -- reinvented those -- a few
years earlier and k-D trees would have
been mine).
Politics: A guy in the clique up there
didn't like me. But it took two levels of
management to fire someone, so he
reorganized to put me under a wuss who
would go along with firing me. They
claimed my research was not publishable
(reviewed by a guy in the clique who
admitted he couldn't read my math but
claimed to have found someone who could
but found nothing wrong with my paper) and
walked me out the door.
The next day the wuss was demoted out of
management. Two weeks later the main
nasty guy was moved down to have him under
an additional level of management and
given a six month performance plan which
he failed. He was demoted out of
management -- lost his corner office,
budget, secretary, and 55 subordinates.
I got a PC and Knuth's TeX and submitted a
paper on my research. Since the paper had
some measure theory in it, e.g., Ulam's
result, much of the computer science
community couldn't read it. But the
journal that offered to review the paper
kept at it; apparently the editor in chief
walked the paper around his campus, to a
CS department to see if the problem was
important and to a math department to see
if the math was correct, and accepted the
paper. He invited me to present at a
conference he was running, but I didn't
want to bother going. The paper was
published. IBM was wrong.
So it appears that I have the world's only
collection, and it's large, of statistical
hypothesis tests that are both
multi-variate and distribution-free, with
some nice properties, with a fast
algorithm, with some confirmation of the
false alarm rate calculations from some
real data, etc.
Asymptotically the critical region can
be a multi-dimensional fractal. Nice.
People should my work. I did give a talk
on the work at the main NASDAQ site in
Trumbull, CT.
IBM didn't pay very well, and cost of
living was high -- I'd always saved money,
even in grad school, but I lost quite a
lot of money working at IBM.
When I joined IBM, it'd just won the Nobel
prize in physics two years in a row and
had a long string of being "the most
admired company in the world". Now I'd
advise anyone just to stay the heck away,
a long way away.
But being pushed out of IBM and age left
me 100% permanently unemployable. I sent
1000+ resumes. Zip, zilch, zero.
If in computing, be sure by age 40,
hopefully by age 35, to have a rock solid
stable career and/or be wealthy. So,
really about have to own your own business
and make it successful.
Well, in part we aimed too high: My wife was brilliant, quite broadly -- yes, Valedictorian, Summa Cum Laude, Woodrow Wilson, PBK, piano, clarinet, voice, prizes in cooking, sewing, raising chickens. But her family had her try to be perfect and to dedicate herself to saving the world.
She wanted a Ph.D. in mathematical sociology to do social engineering of social change to save the world and, thus, get her praise, acceptance, and emotional and financial security. And those concerns filled her plate.
Well, at the roots there some anxiety, from nature and/or nurture, was involved, then some perfectionism and some fear of criticism from powerful, influential people -- net, call it a special case of social phobia. That brought stress which can bring depression. That slows the work and in her case caused more stress and more depression then clinical depression. She was in a clinical depression the day she got her Ph.D. She never recovered and, net, didn't make it.
Took me a while to reinvent and learn the basics of clinical psychology to understand what was going on and what to do about it. I did learn but nearly always too late.
In trying to have a stable job so that I could take care of her, I took a job at IBM's Watson lab, in what we called artificial intelligence (AI) to do monitoring and management of large server farms and networks.
Then IBM got sick and the lab phone book went from 4500 full time names down to 1000 or so. The guy who hired me, a big star who was deliberately ignored by the higher ups, left for greener pastures -- eventually ended up with a nice place in Malibu.
IBM Watson Research was run by a clique of people who stuck together and blocked out everyone else. At one point it appeared that the company HQ in Armonk tried to correct that situation.
Instead of or better than the AI, I had some ideas: Detecting a problem in a server farm or network is necessarily essentially a statistical hypothesis test with Type I (false alarm) and Type II (missed detection) errors. So, want hypothesis tests that keep down the rates of the errors.
Since there's a lot of data readily available and want to exploit it to keep down the rates of the errors, also want multi-variate inputs -- nearly all hypothesis tests have only univariate inputs. Also for such multi-variate data can't hope to know any of the probability distributions so need tests that are distribution-free.
I don't think there were any such tests, so I dreamed up some. I used some group theory, summed used the classic S. Ulam, guy on the left in
http://www-history.mcs.st-and.ac.uk/BigPictures/Ulam_Feynman...
result LeCam called tightness (see P. Billingsley, Convergence of Probability Measures), and was able to permit selecting false alarm rate in advance and then getting that rate exactly. Asking for all of the Neyman-Pearson result would take more data than we had any chance of having, but there was a somewhat useful sense in which my work gave asymptotically, for any selected false alarm rate, the highest possible detection rate with that false alarm rate.
I cooked up some synthetic data that was challenging -- the critical region was something like the red squares on a checkerboard in several dimensions. My work did fine.
I got some data from a cluster of computers at Allstate, wrote some prototype software, and confirmed the false alarm rate empirically. And I cooked up an algorithm to make the computations nicely fast (in part I used k-D trees -- reinvented those -- a few years earlier and k-D trees would have been mine).
Politics: A guy in the clique up there didn't like me. But it took two levels of management to fire someone, so he reorganized to put me under a wuss who would go along with firing me. They claimed my research was not publishable (reviewed by a guy in the clique who admitted he couldn't read my math but claimed to have found someone who could but found nothing wrong with my paper) and walked me out the door.
The next day the wuss was demoted out of management. Two weeks later the main nasty guy was moved down to have him under an additional level of management and given a six month performance plan which he failed. He was demoted out of management -- lost his corner office, budget, secretary, and 55 subordinates.
I got a PC and Knuth's TeX and submitted a paper on my research. Since the paper had some measure theory in it, e.g., Ulam's result, much of the computer science community couldn't read it. But the journal that offered to review the paper kept at it; apparently the editor in chief walked the paper around his campus, to a CS department to see if the problem was important and to a math department to see if the math was correct, and accepted the paper. He invited me to present at a conference he was running, but I didn't want to bother going. The paper was published. IBM was wrong.
So it appears that I have the world's only collection, and it's large, of statistical hypothesis tests that are both multi-variate and distribution-free, with some nice properties, with a fast algorithm, with some confirmation of the false alarm rate calculations from some real data, etc.
Asymptotically the critical region can be a multi-dimensional fractal. Nice.
People should my work. I did give a talk on the work at the main NASDAQ site in Trumbull, CT.
IBM didn't pay very well, and cost of living was high -- I'd always saved money, even in grad school, but I lost quite a lot of money working at IBM.
When I joined IBM, it'd just won the Nobel prize in physics two years in a row and had a long string of being "the most admired company in the world". Now I'd advise anyone just to stay the heck away, a long way away.
But being pushed out of IBM and age left me 100% permanently unemployable. I sent 1000+ resumes. Zip, zilch, zero.
If in computing, be sure by age 40, hopefully by age 35, to have a rock solid stable career and/or be wealthy. So, really about have to own your own business and make it successful.