The title is an extension of the name of a novel by Jane Austen. Sensibility
and feelings might be more what I want to discuss in this blog.
What I want to provide some viewpoints in this blog is the relationship
between sensibility and feelings.
A life with only feelings is dangerous, feelings without the restraint of
sensibility can lead a person astray and on wild tours. I think most persons
have experienced this at some point in their life.
So let's consider life with only sensibility.
Oftentimes one considers science to be based on sensibility and that one
can only find truths with hard facts. This is not my experience in life. Hard
facts and scientific reasoning can get us very far for sure. But there is a limit
to this. There are many things that are so complex that it is impossible to
(at least for me) understand things fully. This is quite obviously true in many
human relations, but also true in trying to understand complex systems.
I remember that I once had a benchmark setup where I tested a highly parallel
update benchmark in a very large cluster setup. What I saw in benchmark
numbers was a strange wave. The performance in the cluster varied between
70M reads per second and 140M reads per second, but it was never stable.
It behaved somewhat similar to a sinus curve with a frequency of a few minutes.
The cluster had about 100 computers and about 5000 CPUs. Thus the cluster
was mathematically a very complex model with thousands of queues that
were interconnected and affected each other. In addition service times in
those queues were dependent on the queue size (TCP/IP messages are handled
more efficiently with bigger packet sizes).
Thus to try to understand this model mathematically could potentially be possible
to some extent, but it was outside of my competence level and I have studied
mathematical models of queueing networks at a Ph.D level so would consider
myself as fairly competent in this area.
The other option to analyse this model would be to perform a simulation that
is close enough to the reality and thus be able to get a better understanding of
the system behaviour. This was a possibility but would probably be an
undertaking of at least a few months.
So what is one to do then in lack of a sensible approach to solving the problem.
This is where I think feelings comes in an important part of understanding the
world around you. My thinking process in those situations is to think about
the system and basically just think and think and contemplate what is going on
in the system.
My thinking about it like this came up with an idea. I started thinking about
traffic jams in a major city and how they can behave the same way where you
go from very slow to very fast in a similar fashion with a similar frequency of
a few minutes.
I concluded that traffic jams are often caused by people trying to drive too fast
and thus causing too many cars to enter a limited section of the road and thus
causing a jam in that section. After that section has been passed the traffic will
usually flow nicely again. Sometimes those traffic jams disappear without any
reasons why they would disappear.
The solution to the traffic jam is that every car has to slow down also in the
areas where there is no traffic jam.
So the question is then, could this approach also work in the computer system
that I worked with.
Obviously this was a theory that I came up with, it was not based on clear
evidence, it was not necessary a sensible approach that things will go faster
by slowing down. I refer to this as a feeling, a hunch that things could be
in a certain way.
So what does one do then when there is no sensible solution, but there is a
feeling that one might have a solution. This is where experimentation comes
in.
In this case I developed a new feature in NDB Cluster that implemented a
new configuration parameter MaxSendDelay. What this feature does is that
it delays sending messages until they have queued for some time.
The effect of this is that more packets to the same destination can arrive
and thus packet sizes can increase and we can improve the flow through
the system.
I implemented this and it worked, the performance jumped to 200M reads
per second and the performance was stable.
So now that I have tested and verified that the solution worked, is the
solution now sensible. Well, it is sensible to use the solution since it
at least works in this particular experiment. Have I thereby understood
all the reasons behind why it behaves as it does? No, I have not, I am still
unable to prove or fully explain why this approach solves the problem.
So sensibility got me a bit on the way, but to move on from the sensible
approach I needed to use feelings. After applying feelings I needed to
perform experiments to verify that my feelings was correct. But even
after applying an experiment to my feelings I still have lots of work
to find the reasons behind why it works as it does.
So in my work I have found that I need to use this three-thronged approach
to extend my understanding.
First learn everything there is to learn through other's experience, thus
ensure that all the sensible approaches in the area is known to you.
When you face an issue where there are no known sensible solutions. Apply
your feelings and contemplate and consider what could be the solution.
Consider the problem using a mind model in your brain and look for other
similar problems that you might have experienced and at least found theories
of how it works.
Once you decided on a theory, apply experiments to verify your theory.
If the experiment is successful then at least you have started to understand
the issues at hand. But there is still more work ahead to turn it into a sensible
model that can be used by others to extend their understanding.
Now this model has worked for me in my work life. Actually it has worked
well for me also in my religious life.
I came across the Church of Jesus Christ of Latter-Day Saints more than
30 years ago. I met a number of missionaries that presented their faith to
me.
I followed the above principles in evaluating their message. I started by reading
a lot about their religion, trying to understand whether their approach to life
seemed sensible.
Obviously as with any religion there is a point that one has to apply faith in order
to move on. This is actually true in work life as well, but the faith required to
change your life is a bit more than the faith required to try out an experiment on
a software system.
Thus the feelings required in a decision about religious matters need to be a bit
more profound. In my case it was a very clear feeling that what I read about was
true. This feeling was very convincing.
Armed with this strong feeling of conviction one can wonder if all is done?
For sure not, the step of experimentation is still ahead. Every commandment
in a religion comes with two parts. One part is the commandment, the second
part is the promised blessing from the commandment.
Thus every commandment in a religion can be experimented on. When such
an experiment brings you the blessings you have gained some understanding.
But the road is still long ahead to understand everything fully.
and feelings might be more what I want to discuss in this blog.
What I want to provide some viewpoints in this blog is the relationship
between sensibility and feelings.
A life with only feelings is dangerous, feelings without the restraint of
sensibility can lead a person astray and on wild tours. I think most persons
have experienced this at some point in their life.
So let's consider life with only sensibility.
Oftentimes one considers science to be based on sensibility and that one
can only find truths with hard facts. This is not my experience in life. Hard
facts and scientific reasoning can get us very far for sure. But there is a limit
to this. There are many things that are so complex that it is impossible to
(at least for me) understand things fully. This is quite obviously true in many
human relations, but also true in trying to understand complex systems.
I remember that I once had a benchmark setup where I tested a highly parallel
update benchmark in a very large cluster setup. What I saw in benchmark
numbers was a strange wave. The performance in the cluster varied between
70M reads per second and 140M reads per second, but it was never stable.
It behaved somewhat similar to a sinus curve with a frequency of a few minutes.
The cluster had about 100 computers and about 5000 CPUs. Thus the cluster
was mathematically a very complex model with thousands of queues that
were interconnected and affected each other. In addition service times in
those queues were dependent on the queue size (TCP/IP messages are handled
more efficiently with bigger packet sizes).
Thus to try to understand this model mathematically could potentially be possible
to some extent, but it was outside of my competence level and I have studied
mathematical models of queueing networks at a Ph.D level so would consider
myself as fairly competent in this area.
The other option to analyse this model would be to perform a simulation that
is close enough to the reality and thus be able to get a better understanding of
the system behaviour. This was a possibility but would probably be an
undertaking of at least a few months.
So what is one to do then in lack of a sensible approach to solving the problem.
This is where I think feelings comes in an important part of understanding the
world around you. My thinking process in those situations is to think about
the system and basically just think and think and contemplate what is going on
in the system.
My thinking about it like this came up with an idea. I started thinking about
traffic jams in a major city and how they can behave the same way where you
go from very slow to very fast in a similar fashion with a similar frequency of
a few minutes.
I concluded that traffic jams are often caused by people trying to drive too fast
and thus causing too many cars to enter a limited section of the road and thus
causing a jam in that section. After that section has been passed the traffic will
usually flow nicely again. Sometimes those traffic jams disappear without any
reasons why they would disappear.
The solution to the traffic jam is that every car has to slow down also in the
areas where there is no traffic jam.
So the question is then, could this approach also work in the computer system
that I worked with.
Obviously this was a theory that I came up with, it was not based on clear
evidence, it was not necessary a sensible approach that things will go faster
by slowing down. I refer to this as a feeling, a hunch that things could be
in a certain way.
So what does one do then when there is no sensible solution, but there is a
feeling that one might have a solution. This is where experimentation comes
in.
In this case I developed a new feature in NDB Cluster that implemented a
new configuration parameter MaxSendDelay. What this feature does is that
it delays sending messages until they have queued for some time.
The effect of this is that more packets to the same destination can arrive
and thus packet sizes can increase and we can improve the flow through
the system.
I implemented this and it worked, the performance jumped to 200M reads
per second and the performance was stable.
So now that I have tested and verified that the solution worked, is the
solution now sensible. Well, it is sensible to use the solution since it
at least works in this particular experiment. Have I thereby understood
all the reasons behind why it behaves as it does? No, I have not, I am still
unable to prove or fully explain why this approach solves the problem.
So sensibility got me a bit on the way, but to move on from the sensible
approach I needed to use feelings. After applying feelings I needed to
perform experiments to verify that my feelings was correct. But even
after applying an experiment to my feelings I still have lots of work
to find the reasons behind why it works as it does.
So in my work I have found that I need to use this three-thronged approach
to extend my understanding.
First learn everything there is to learn through other's experience, thus
ensure that all the sensible approaches in the area is known to you.
When you face an issue where there are no known sensible solutions. Apply
your feelings and contemplate and consider what could be the solution.
Consider the problem using a mind model in your brain and look for other
similar problems that you might have experienced and at least found theories
of how it works.
Once you decided on a theory, apply experiments to verify your theory.
If the experiment is successful then at least you have started to understand
the issues at hand. But there is still more work ahead to turn it into a sensible
model that can be used by others to extend their understanding.
Now this model has worked for me in my work life. Actually it has worked
well for me also in my religious life.
I came across the Church of Jesus Christ of Latter-Day Saints more than
30 years ago. I met a number of missionaries that presented their faith to
me.
I followed the above principles in evaluating their message. I started by reading
a lot about their religion, trying to understand whether their approach to life
seemed sensible.
Obviously as with any religion there is a point that one has to apply faith in order
to move on. This is actually true in work life as well, but the faith required to
change your life is a bit more than the faith required to try out an experiment on
a software system.
Thus the feelings required in a decision about religious matters need to be a bit
more profound. In my case it was a very clear feeling that what I read about was
true. This feeling was very convincing.
Armed with this strong feeling of conviction one can wonder if all is done?
For sure not, the step of experimentation is still ahead. Every commandment
in a religion comes with two parts. One part is the commandment, the second
part is the promised blessing from the commandment.
Thus every commandment in a religion can be experimented on. When such
an experiment brings you the blessings you have gained some understanding.
But the road is still long ahead to understand everything fully.
