We're a free community of over 18,000 Texas technology professionals here to network and promote our local tech scene.
If you're working in semiconductors, hardware, software, or IT in Texas, join us!
Next Project Will Be Different!
Submitted by Jane Prusakova on Thu, 07/23/2009 - 1:18pm.
::
Share this on Facebook!
With this latest economic crisis another area of science is coming to an understanding that theories based on knowledge about people are both different and more important than numbers-based theories. The Economist recently published an article discussing how the latest accepted hypothesis on efficient markets has lead the global economy to the today's sorry state.
An opposing theory of behavioral economics has been around for nearly as long as the efficient-markets idea, but did not have much adoption in the dealings of the Wall Street.
Agile methodology is based on a behavioral hypothesis, its recommendations come from real-life observations of people involved in software development. Requirements change, bugs happen, estimation takes a lot of time and is error-prone, and people generally want to do a good job - these happen in most if not all development projects.
Yet, there is plenty of temptation to declare that the next project will have its requirements decided upon from the start, that the team is good enough to not create many bugs, and the estimate that was created in the first phase of the project will be adhered to up to the delivery date. Time and time again, the slow and careful estimate attempts to save time on painful integration and test, and accommodate client's unavailability (and lack of interest) for the project. The resulting disaster is then blamed on programmers who created too many bugs, and first-line managers who allowed bad and lazy people on the team and did not [micro]manage enough.
Granted, some (if few) waterfall-ish projects are successful, and it takes more than one project failure to realize that something is wrong with the system. But must the industry wait for a global collapse, like the Wall Street did?
- Jane Prusakova's blog
- Login or register to post comments
In the realm of software development it is true that waterfall project success is abysmal. In other areas and sectors that is not the case... dependent to some degree on our definition of project success.
Waterfall project failures are reported by statistical means based on the "old school" criteria of time (schedule) and cost with a nod to quality as defined in the specs or requirements. We discussed this on enweave with reasonable detail in the blog titled Defining Project Success.
One of the mantras I repeat often to organizations and project teams is this: If a process can not be correctly executed 90+% of the time by smart, well intentioned people, it is a bad process. "Method" or "approach" may be substituted ad lib for "process," where appropriate. A corollary is: "If a correctly executed process fails to produce the desired outcome(s), it is a bad process."
The same might be said for models... and models are the point of the article referenced in the Economist, not methods, approaches, or processes. In one sense -- an important one -- philosophies and models are very similar. To the extent a model (or philosophy) represents a valid representation of actual, real, measurable conditions or events at any given point in time, it is useful. As the model diverges from testable reality, it becomes less useful.
At this point it is important to draw a distinction between the Agile philosophy and its implementation or executing within organizations. The two are not necessarily the same... most often they are not the same. The philosophy is the core of this discussion.
The problem statement in the Economist article is somewhat different than our problem with software development methodology. As Myron Scholes stated, "...much of the blame for the recent woe should be pinned not on economists’ theories and models but on those on Wall Street and in the City who pushed them too far in practice." In other words the theories (or hypotheses) still have credibility but their practice is flawed.
Still, the comparison has some merit within software development especially in the need for behavioral attributes within our models. "Creative" endeavors, such as software development or graphic design, have core differences when compared with engineering bridges or constructing highways. While both the creative side and the less creative side both require behavioral attributes, the implementation of them will differ considerably. For example, waterfall development methods continue to be successful in bridges and roads and will probably continue to be successful.
It is important to note at this point, and referencing back to the Economist, that the "collapse" leading us into this recession was based entirely on real estate and mortgages. It was aggravated by computer driven sell/buy models based on erroneous input and risk logic. It was not the collapse of a overall market philosophy nor of any particular hypothesis, theory, or model.
In a more general sense our "collapse" has already occurred and we are working out of it... or attempting to do so. It happened back in the 50's when US companies chose to rely on "planned obsolescence" and similar approaches while letting quality slip into the category of "other considerations." There are literally thousands of examples from autos to cameras and steel... too numerous to cite.
For those who remember, or who have studied quality history, Japanese products (as one example) manufactured prior to the late 1950's were synonymous with shabby, plastic, low quality, and throw-away. In post-Deming, Juran, Ishikawa, Kaizen Japan, that situation turned 180 degrees. Japan took the market away from US producers by manufacturing similar or higher quality goods at lower price points. In most sectors they remain the quality leader to this day. The work of Deming, again as just one example, didn't even begin to become accepted in the US until near the time of his death (1993).
The result is that organizations in the US have either completely abrogated to the competition or are still in catch-up mode compared to organizations (manufacturing and other sectors) in other countries in terms of quality to price. US goods are not of intrinsically higher quality compared to goods and products in the same price range from other countries. This is not an "anti-US" sentiment, it is simply a true statement.
In this country we are still far behind others in the overall application of quality techniques. It is just as true in software development as it is in automobiles and cameras. The Agile philosophy is a big step forward toward providing a model of integration for those techniques.
Most people and organizations in the US still believe (erroneously and despite fifty years of evidence to the contrary) that higher quality always comes at higher price. Models and approaches that drive higher quality outputs and which beat the competition (using other methods) at lower price are available, proven, and in dire need of adoption. Agile is something that can go a long way toward dispelling that "higher quality = higher price" myth and lead to significantly higher quality results in software development.
William W. (Woody) Williams
Project Management Consultant
| Blog | Twitter |
w3src Consulting
Thank you for a well-reasoned response.
Totally agree with your mantra about what makes a bad process. The process is a tool to make work easier and more productive (i.e. achieve higher quality at lower cost). If the process is costly (i.e. competent well-intentioned people can not follow it) or does not achieve good result when followed, this process is not a good tool.
I am not sure that the lack of quality that has plagued US industry for several decades is in any way connected to the software industry issues. I do not know of many projects that deliberately set out to build a poor solution, to miss the delivery date, or to deliver a non-working product.
The Economist article discusses how Wall Street embraced efficient-markets economic theory, that eventually led to the collapse. The theory that was used assumed that people make rational decisions that lead to efficient markets, but people and markets are not exactly rational. Models developed based on efficient-markets hypothesis disregarded certain types of economic risk, causing the global problem once this unaccounted-for risk built up.
Similarly, widely accepted Waterfall approach appears rational - and fails more often than not, in large part because it does not adequately monitor project state and risk exposure. Agile does not guarantee quality product, but it's processes provide a way to monitor and adjust development to achieve desired result - whatever is the desired result. In particular, Agile processes give visibility into the project risk, and thus guarantees more predictable results.
Jane Prusakova
Software Architect & Developer
My blog
Second paragraph: Didn't intend to imply that quality, especially quality in software projects, is a deliberate decision or approach on the part of project teams or organizations in the sense that everyone gathers around the table and agrees to produce a non-working product ;~) It is primarily the lack of adoption of process that lead to high (or higher) quality products that is the underlying cause. And, to bring that full circle, the lack of adoption is driven largely by the perception that "higher quality = higher cost." So, there is a psychological barrier as well as a lack of process/tools.
There is a caveat to the above. Software development efforts are constantly under pressure to "cut this" and "speed up that." Even though we (project and program managers) clearly state the obvious point that quality will (not may; will) suffer as a consequence, many times the decision is made to do so anyway. This is an example where the decision is deliberate and intentional. It happens a lot... perhaps even "all the time."
Third paragraph: My reading of the article in The Economist is a little different and I think the conclusion drawn by The Economist on the death of EMH is not quite in alignment with yours either. The Economist says it this way (emphasis added), "That is why many people view the financial crisis that began in 2007 as a devastating blow to the credibility not only of banks but also of the entire academic discipline of financial economics. That verdict is too simple." I agree, that is neither the correct conclusion nor an accurate interpretation of events.
EMH is no "silver bullet" hypothesis nor has it ever been. As The Economist notes, people have been shredding it since the ink was dry on the initial presentation ;~) Certainly there are staunch advocates but there are also still members of The Flat Earth Society around as well.
In fact behaviorists and EMH adherents in the academic mainstream have been converging rather than diverging over the past decade and both are picking up points from other theorists as well... something The Economist explains in great detail.
Reading The Economist article, to me, points out how these models evolve and change as new or different knowledge emerges. It isn't the "win/lose" scenario of one hypothesis defeating another, it is instead an integration and convergence of thinking through collaboration and sharing leading to newer model versions.
The misuse of the model by financial institutions leading to the mortgage meltdown may provide some impetus to the convergence or integration but it is not the cause of it and is, in fact, unrelated. The EMH model didn't "fail," it was misused. Now that may be a rather fragile distinction in the face of plummeting home values and job loss but it is still a rather important one. It is a distinction The Economist makes as well.
The "model" that led to the collapse of the far flung mortgage backed security bubble was more based on arrogance, greed, and stupidity than EMH or a lack of behaviorist integration in the model.
If the risks of those derivatives had been calculated honestly and correctly, the "bubble" would never have formed. Those derivatives sold like proverbial hotcakes because they were highly rated -- the old "safe as houses" convention. With correct ratings, there never would have been as much pressure on the demand side and those instruments would only be a footnote in financial history.
As Scholes remarked, "Apparently, a lot of the models used for structured products were pretty good, but the inputs were awful.” The Economist then says, "Indeed, the vast majority of derivative contracts and securitisations have performed exactly as their models said they would. It was the exceptions that proved disastrous."
Risk factors are among those "inputs." And, as The Economist points out, it was only in mortgage backed derivatives ("the exceptions") where flawed inputs led to trouble.
So, how does this get back to software projects?
Risks.
If the risks of poor quality output were correctly and accurately established by businesses and organizations and applied both in terms of internal and external software development projects, there would be much more emphasis on quality.
Just focusing on the internal: It's still difficult for many organizations to get their head around the concept that an internal software project, for example, is a "product." In other words, if the same quality processes that the organization applies to creating, marketing, and producing their salable product (whatever that may be) were applied to internal software development efforts (which is almost never the case), the result would be a much higher emphasis on quality in these efforts as well as the tools, resources, and processes to make it work.
This is not simple. Neither is it trivial. It is a huge undertaking but the results are worth it.
William W. (Woody) Williams
Project Management Consultant
| Blog | Twitter |
w3src Consulting
Woody,
I see a parallel in how Wall Street took an economic theory that appeared rational and fed the models less-than-honest inputs, without questioning whether the model is accounting for all relevant risk factors while it was making them rich. The theory might not have been that bad, but the models did have a fatal flow - they allowed exceptions that behaved unpredictably and disastrously.
A software development industry for a long time chugged away using various waterfallish models, with less-than-honest inputs (i.e. our people do not create bugs). When projects fail, the tendency has been to blame the particular people involved, rather than the process/model.
More recently economists have been coming up with models that incorporate both behavioral and rational ideas - The Economist article mentions Andrew Lo of the MIT. Successful Agile teams find themselves spending some time on planning, estimation and design, and doing more testing toward the end of the project than at the beginning - even though the "pure" theory says that no planning is needed and design should just "emerge".
Jane Prusakova
Software Architect & Developer
My blog
Jane,
I understand and have great symphathy for your viewpoint. However (you knew it was coming ;~)...
It wasn't the models used that failed to account for the risk factors. It was people feeding the model incorrect, flawed, and frankly spurious information.
Please note in the article the fact that all other speculative and derivation models performed within expected parameters using the same models. This is a key point.
It was only mortgage backed derivatives that were fed flawed inputs and therefore (as expected) the model failed to produce accurate results but only for these specific derivatives. Another very key point.
Scholes states this in a very politically correct manner saying that financial analysts input data that reflected a, "view of the world that was far more benign than it was reasonable to take, emphasising (sic) recent inputs over more historic numbers.”
In other words they fudged the input data to reflect lower risk ("more benign") and greater return ("recent inputs"). This is greed and stupidity, not an academic hypothesis ;~)
A model is no better than its inputs -- the model predicted events and conditions based on flawed data for mortgage backed derivatives. Where the input data was not flawed, other derivatives and security types, as The Economist says, "...performed exactly as their models said they would. This is definitive.
Where in all other cases the model, with correct inputs, produces exact "as expected," results there is no issue with the model, it's basis, or algorithms.
This "meltdown" is not something that can be laid at the feet of any model, hypothesis, or theory. It is the result of people -- their greed and stupidity.
As much as we might like to credit the disaster to EMH, and I'm no ardent fan of the hypothesis as stated originally, the facts simply don't support that conclusion.
That's not to say that EMH is perfect, beautiful, or Truth (capital T). It is not and that is also known. It's just that, in this particular case, the EMH hypothesis and the models based on it are not accountable for the meltdown.
William W. (Woody) Williams
Project Management Consultant
| Blog | Twitter |
w3src Consulting
Woody, thank you for sharing your viewpoint - but here goes... ;)
A model is responsible for its output, not matter what the inputs. A model that works great most of the time, but occasionally fails disastrously is dangerous. Such a model is in many ways worse than the one that fails more often but in a more predictable way.
Mr.Scoles blames the users, but The Economist article mentions two hedge funds that Scoles founded that both tanked. Supposedly, Scoles practices what he preaches and used the EMH model he defends in the article to evaluate assets his funds owned, and the model did not perform for him and his very competent staff either.
The models adopted by Wall Street hid the risk building up in the system from the bankers' view. That includes EMH that allowed for unpredictable and disastrous exceptional outcomes, and VAR models, that assumed constant price correlations between assets.
Jane Prusakova
Software Architect & Developer
My blog
First, I'd like to make sure we're talking about the same thing. I believe there may be a confusion in terms.
To me, a project manager with some background in financial services, a past holder of securities licenses, and other salient experiences, here's what a financial model looks like... It's from Wikipedia and I am in agreement with the general description.
"Financial modeling is a general term that means different things to different users. For some, it means the development of a mathematical model to predict a fair equilibrium price for an asset. For others, it means the development of a mathematical model and the associated computer implementation to simulate scenarios of financial events, such as asset prices, market movements, portfolio returns and the like. Or it might mean the development of optimization models for managing and controlling the risk of a financial investment."
I have seen all "flavors" mentioned above used in various organizations. I've helped develop some ;~) and this is what I, The Economist, and Scholes are referring to as "model" or "models."
EMH is not a model in this sense. It is an economic theory or, more precisely, an hypothesis... the "H" stands for "hypothesis" ;~) Models may be built based in whole or in part on EMH but EMH is not a "model" and the article in The Economist is not using the term "model" as reference to EMH. Neither is Myron Scholes.
So, if you are saying that an economic theory or hypothesis such as EMH should be an accurate gauge, predictor, or reflection of actuality regardless of inputs, I agree. Wholeheartedly; complete agreement.
In the case of financial models however (always the "however" ;~), they are all dependent on the quality and correctness of their inputs for the quality of their output. There are also internal algorithms, calculations, and assumptions but they do not "stand alone" like an economic theory or hypothesis even though models may be, in whole or part, based on them or assumptions derived from them.
Scholes is not defending EMH, he is defending the financial models used for securities and derivatives -- primarily the ones used for pricing and risk.
I hope that makes sense and clarifies the reasoning and points made in my previous posts.
William W. (Woody) Williams
Project Management Consultant
| Blog | Twitter |
w3src Consulting
A good mathematical model (with or without its associated computer implementation), behaves true to the hypothesis of the economic theory on which it is based. "Good" inputs to the system produce results that fit the theory, and "bad" inputs should break the model - produce a contradiction in terms, values outside the expected range, etc..
Mr Scholes says that the models used by Wall Street were good, i.e. the software implemented the central hypothesis of EMH well. EMH is a new theory, developed after the war, largely based on data generated during a period of economic stability. The article quotes Scholes: «Financial firms plugged in data that reflected a “view of the world that was far more benign than it was reasonable to take, emphasising recent inputs over more historic numbers,” » But that is the view of the world and the data that EMH is built on, even if in practice "the benign view of the world" has been taken to the extreme.
At the same time, Mr Scholes has developed the formula widely used by Wall Street to price securities, so it is quite understandable that he chooses to defend the model, and blame the input.
Jane Prusakova
Software Architect & Developer
My blog
Jane,
I have read and re-read the article in The Economist trying to understand on what basis your conclusions are drawn without success.
For example the Scholes quote, “view of the world that was far more benign than it was reasonable to take, emphasising recent inputs over more historic numbers,” is not referring in any way to EMH. It is referring to the, arguably, criminal way in which analysts manipulated inputs to their financial pricing models.
The "benign view of the world" Scholes refers to isn't based on EMH... as he says, the analysts emphasized recent inputs over historic numbers. "Recent inputs" is not EMH, it is the close to criminal manipulation of the numbers to derive lower risk and higher security prices. Read that paragraph of the article carefully, it is important.
The fact that all other derivatives (excluding mortgage backed derivatives with criminally skewed inputs) performed according to the financial models expectations is simply a showstopper for your interpretation of the article.
Further, Scholes is no ardent defender of EMH. The article outlines many areas where he has and currently holds EMH as wrong. Thayer, on the other hand -- the prime behaviorist, "defends" portions of EMH.
As the article points out, EMH and other theories including more behavioral ones are converging. The initiating cause of the fiasco with mortgage backed derivatives has nothing at all to do with EMH or any other financial theory. It has everything to do with the misuse of financial models -- greed and stupidity.
The inability of economists and financial analysts to see where that fiasco was going (downstream impact) does have a lot to do with theories, hypothesis, and EMH. However, EMH was not the root cause of the crisis, only a factor in misunderstanding the impact.
William W. (Woody) Williams
Project Management Consultant
| Blog | Twitter |
w3src Consulting
Woody,
Numbers manipulation may have been criminal, but the fact remains that EMH-based models allowed the crisis to happen. The article mentions several ways in which EMH has been criticized by economists other than Scholes: Sanford Grossman and Joseph Stiglitz pointed out a paradox in 1980, Franklin Allen discussed how little attention economics theory paid to financial institutions in 2000, behavioral economists have been skeptical about the assumption of inherent market rationality in EMH since EMH first came out.
The economic science has moved on past EMH, toward blending EMH ideas with behavioral approach, as the article mentions, but the Was Street continued to apply EMH-based models until 2007. These are the models designed on the EM hypothesis as defined in the 70s.
The fact that other derivatives behaved as predicted by EMH in the last few decades does not show that EMH is correct. A stopped clock tells true time twice a day, too.
I agree, EMH, being a scientific hypothesis, is not the cause of the crisis. But Wall Street putting too much trust in the EMH was.
Jane Prusakova
Software Architect & Developer
My blog
The "other derivatives" referred to were not historic in the sense of decades. They were concurrent derivatives -- concurrent in time with and using the same financial models as the mortgage derivatives.
As I read The Economist, the cause of the mortgage backed derivative meltdown was manipulation (misrepresentation) of data inputs.
There are other problems with Wall Street, financial firms, and "the system" in general that exacerbated the meltdown. Those are being addressed.
On those and any further points, I agree to disagree at this point ;~)
William W. (Woody) Williams
Project Management Consultant
| Blog | Twitter |
w3src Consulting
Woody,
thank you for the discussion.
"There are other problems with Wall Street, financial firms, and "the system" in general that exacerbated the meltdown. Those are being addressed." - I certainly hope so. However, greed can not be addressed - it'll always be there. We just need a system (a model) that keeps it in check.
Jane Prusakova
Software Architect & Developer
My blog