Models, Methods, and Software Research at Olde Camp NRTS: Part 1

by Dan Hughes

This is the second of a few posts that summarize some of the activities within the New Analysis Development Activity (NADA) group in the early- to mid-1970s.

They were best of exciting times. We were decades ahead of the world.

They were the worst of times. Management at the USNRC dismissed the very significant number of years of experience and expertise represented by the group members. That same USNRC had paid for the advances that they threw out the window.

Update Tuesday June 22, 2017: This post might make more sense after reading Part 0

Update Tuesday April 18, 2017: I have corrected the nomenclature based on information from Chuck Solbrig.

The SLOOP Project 
The primary objective of the development efforts was a model and software that could be applied to analyses of the fluid flow, heat transfer and associated physical phenomena and processes on the primary side of a nuclear reactor powered electric generating system. These are commonly referred to as loops. The working name for the new code was SLOOP. The S  in SLOOP came from Serrated Seriated, Chuck’s nomenclature for models of mixtures in which the various constitutes occupy separate, distinct regions of the flow field. The mixtures are not formed by interpenetrating constitutes which is the classic Continuum Mechanics model of mixtures. While the nomenclature is clearly appropriate and correct, it has never caught on.

The fact that the UVUT model equation system had complex characteristics was discovered early in the 1970s in the course of the SLOOP code development. Details of the discovery are given in The History of Multiphase Computational Fluid Dynamics. In particular, in that summary, Bob recalls the details of the saga of getting this paper published. I recall Dimitri coming into the office one morning shaking his head, looking down, and quietly muttering, “The characteristics aren’t real”. Real characteristics being a necessary property of well-posed hyperbolic Initial Boundary Value Problems. While all of us had been introduced to the concept of the characteristics of Partial Differential Equations (P R Garabedian, 1964), I personally had never had an occasion to consider the deeper meanings of this property of PDEs. A significant oversight on my part. Here I give some of my personal recollections of the activities at that time.

It is noteworthy that within Continuum Mechanics, frequently labelled an exact science, development of a satisfactory formulation of the fundamental equations for mixtures of interpenetrating constitutes followed an evolutionary path from the 1950s or 1960s (Clifford Truesdell) to well into the 1980s (Bowen et al.) and 1990s. Some are not so certain that the formulation is yet complete (Trapp and Welch, 2010). Continuum Mechanics papers on mixtures continue to appear to this day in 2017, and some are now addressing the matter of the lack of interpenetrating constitutes (Muller, 2017). Even the characteristics problem has been picked up by mechanics who work in the continuum paradigm. (Or is that an axiom in the sense that it can’t be proven?)

An Engineering Science approach, founded upon firm mathematical principles, within the engineering communities seems to have been initiated with the work of D. A. Drew (1971). Thousands of peer-reviewed papers, beginning in the late 1970s, have appeared in a wide range of journals covering several engineering, and mathematics, disciplines. Papers continue to appear to this day.

Papers in which the many issues associated with the lack of real characteristics in the UVUT two-phase flow engineering models are investigated also continue to appear. Generally, the deep source of, and ramifications of, the lack of real characteristics remains an unresolved matter. And based on its status in the literature and the enormous amount of work that has been applied to the matters it is very likely that resolution may never be attained.

At the time that the discovery was made, the group had a test-bed code up and running, but successful completion of a calculation was elusive. The calculations always blew up with all the characteristics of instability in the numerical solution method; exponential growth of small perturbations. We were all baffled because a “completely implicit” finite difference approach to the differential equations was coded. That in itself was a departure from the methods used in all other codes, and they were all based on the EVET model. At the time, we were prepared to accept the significant computing-time costs of an implicit iterative solution of the nonlinear discrete equations.

A rudimentary matrix solver was used without optimization based on the structure of the discrete equation system, so an un-countable number of operations were being performed that resulted in zero, and long run times. The persistence of the problem also meant that we were getting tons of de-bug printout. The operators were not happy whenever our batch-mode jobs showed up. The best time to get good turn around on a calculation was at night when everyone else had gone home; and at the weekends when no one else came to work.

We were working on the basis that a fully implicit method ensured stability of the solution method. Well, we have to be careful on what, exactly, does fully implicit mean. I recall that the void fraction was updated as the last step at the end of the iterative loop. After weeks of trying various proposed solutions to the problem, Glen Mortensen decided that if the solution for the void fraction was added into the matrix of the solution variables, the calculation would be successful. Maybe Glen had even tried out the approach with some of the software that had been written to investigate the properties of the solution method.

One Sunday evening, after I had done the stuff that I had gone back to the office to take care of, I called Glen at home to get the details of his approach. He outlined what changes I had to make to the coding, they were all simple, and at the same time we ‘turned off’ everything that was not necessary for the test. It was a very straight-forward change, so I made an update deck, as we called them, dropped the cards, and they were indeed IBM punch cards, off at the counter and went home.

The next morning there was my run on the output counter. It had run to completion. When the results were reported to management one of them reportedly said, “This changes everything.”

But by then it was too late, and other facts indicate that there were no actions whatsoever on our parts that could change the situation, to stop the implosion of the team.

To this day, I consider the few years spent on the various tasks, ca. 1971-1975, we undertook in the new code development research and development to be among the more rewarding of my career. We were a young, enthusiastic, confident, and competent group, each of us excited about and fully engaged with our work, and looking forward to construction of brand new, and important, models, methods, and software. I consider the team to have been one of the most knowledge filled that I have encountered in my entire career.

The decisions that were made to get the efforts stopped have been proven beyond any shadow of doubt to be grave mistakes on the parts of those making the decisions. We were decades ahead of the curve and had an deep pool of excellent talent. The NRTS also was at the lead in water reactor safety experimental efforts. The data from tests conducted then are still used for validation of models and software. The tight combination of model development and experimental validation is absolutely the most powerful combination that can be devised for ensuring that the models are correct for their intended applications. Additionally, the other model development efforts ongoing at the NRTS were the aspects that were needed to make a complete safety analysis system. Many of those models and codes are still in used today.

The first calculations of transient, compressible, boiling/flashing two-phase flows with a two-fluid model-equation system done anywhere, were done in Idaho Falls under the SLOOP code project in the early 1970s. That beginning has led to uncountable numbers of reports, papers, and applications over the past 45 years. And the numbers increase by the hundreds, if not thousands, every year.

It is interesting to note that now, over 50 years from those formative years, some versions of these codes, descendants of the original versions, are still in use. And the concepts have been employed to develop models, methods, and software all around the world where nuclear reactor safety is researched, developed, and practiced. Other focus areas have also evolved from these early years; detailed models for fuel mechanical performance, transient 3-dimensional neutronic kinetics, and recently Computational Fluid Dynamics (CFD) and turbulence models. As this history shows, the latter two were actively pursued starting in Idaho Falls in the early- to mid-1970s, about 45 years before they again surfaced in water reactor safety research.

The ZUVT Code
After Chuck and I had settled on a first-order model equation system, one of my initial follow-up jobs was to investigate the thermodynamics, and the energy and mass exchanges of mixtures of the liquid and vapor phase of water, under both thermal equilibrium and non-equilibrium conditions. To this end I built a small code that focused solely on these issues. The there was no motion accounted for and only the mass and energy balances were used. This “model” we labeled Zero Velocity Unequal Temperature, ZVUT. The objective was to get an initial handle on the “energy partition problem”. The first outline of the problem is discussed in the hand-written manuscript that Chuck and I wrote; Two phase flow equations which account for unequal phase velocities and unequal phase temperatures October 1971, Aerojet Nuclear Company, Charles W. Solbrig and E. Daniel Hughes, and in the ANCR report.

At the time, in the early 1970s, very little research had been focused on the con-current flow of the liquid and vapor phases of water under non-equilibrium (Unequal Temperature) conditions. We knew that this aspect of the model would be the limiting aspect relative to attaining closure of the energy equations. The “energy partition problem” continues to be an area of active research to this day; Hewitt (2016). The issue is encountered for both the wall-to-fluid and interphase (phase-to-phase) energy exchange. For the wall-to-fluid case, at steady state conditions, with the wall supplying energy to the fluid, for example, the total heat flux due to the wall will be known. The questions then are what part of the total goes into sensible heating of each phase and what fraction of that going into the liquid phase contributes to evaporation of liquid. The corresponding questions, sensible cooling, and condensation, obtain for the case of a cooled wall. The ZVUT modeling and software was designed to address these matters.

An interesting matter arose at the very first time I attempted to apply the model; the numerical solution method would not maintain a null transient. Null transient cases had been used by the group as a zeroth-order check on the correctness of the coding. The initial and boundary conditions were set so that no subsequent changes would evolve as the equations were integrated out in time. I pounded on the problem for at least a couple of weeks; checking and re-checking the coding and the specifications for the initial and boundary conditions. I finally tracked the problem down to a unit-conversion matter. The Steam Tables being written in Engineering Units and our codes in SI.

To specify the initial conditions, I was using the tabulated water properties that were published by the American Society of Mechanical Engineers, ASME. These were simply known as The Steam Tables. The code was using a representation of these that was in wide use around all the model development efforts going on at the laboratory. I finally discovered that the J conversion factor used by the ASME tables, and the value programmed into the code routines differed in the last decimal place. I was specifying numbers that were not in agreement with those calculated by the code. That change fixed the problem. For me it was a lesson in software debugging.

The problem also illustrated the power of the null-transient concept. That concept proved to be very useful throughout all my software development and was an integral part of many projects over the decades.

Lance Agee interviews and gets job offer
While the work on the SLOOP and SCORE models, methods, and software were among the more enjoyable of my career, that work was directly responsible for, and led to, additional research along the same lines. And that work proved to be an equally exciting and ground-breaking application of advanced models, methods, and software. Here’s an introduction to the story about Lance Agee and the Electric Power Research Institute (EPRI). EPRI played a very significant role in research and development for water reactor safety in general, and in the succes of some of the spin-off companies around Idaho falls.

Lance Agee sent his resume to us sometime in ca. 1974. At the time, Lance worked for Combustion Engineering (CE) in Connecticut. He came out for an interview and was given high marks by everyone involved and so got a job offer. Lance was also excited about our concepts and efforts. He had been pushing for advancing the models and methods used by CE. We also thought it might be a good idea to have a team member who knew exactly what was going on out in the real world of nuclear electric power production.

Lance accepted the offer and put his house on the market. It was late in the year, maybe October, November, or December and of course selling the house proved to be a difficult matter. Someone in Camp NRTS Management decided that it was taking far too long for Lance to report for work and demanded that he show up by the end of December 1974 or the offer would be revoked. He was not successful in selling the house, he didn’t report for the job, and the offer was.

Lance will re-appear later in these notes. Although Lance missed an opportunity to be a part of our most excellent team, it will be discovred that Lance scored a home run for two reasons; (1) the dissolution and dispersion of the NADA group only six months after he would have reported for work (dodged that), and (2) the research environment at EPRI allowed him the opportunity to fully implement his vision of developing advanced Thermal-Hydraulic models, methods, and everyday-engineering software tools for the Nuclear Electric Power Industry in the United States. Lance fully took advantage of, exploited, and successfully utilized that opportunity, including breaking into the international market for his goods and services.

___________

Footnote Diversion / Distraction: It’s a small world
Some years later, 30 years later in 2004 in fact, while Biker Babe Mary and I were on a cross-country Motorcycle Road Trip, I ran into Lance’s brother. We were crossing the desolation (a state of complete emptiness or destruction; anguished misery or loneliness) that is Nevada on US 93 when we stopped for a break in Rachel, NV.

As I was leaving the cool air-conditioned store to re-enter the superheated desolation, out of the corner of my eye our inherent rapid survival instinct of facial recognition when given only a tiny fraction of the complete scene automatically kicked in, and my brain said, “You know that guy.” It was only many miles later down the road that my brain, having absolutely nothing else to do on the perfectly flat and straight roads said, “That was Lance’s brother.”

That occurred because Lance’s brother had flown up to Idaho Falls in a small private airplane to pick up Lance and take them to The Ranch, from which Lance had escaped after deciding that he did not want to punch doggies for a living, back in Nevada. I had met the brother for a couple of minutes at the Idaho Falls airport way back in ca. 1974. The fact that his brother is as tall as Lance, or maybe taller, might have assisted my survival instinct mechanisms. Not to mention that the brain being keenly aware of where we were located very likely kicked into hyper-sensitive survival mode. The brain is amazing !

The SCORE Code
But next we need to visit the SCORE code, and the related SPLEN code. The SCORE and SPLEN codes, Serrated Seriated Core Code and Serrated Seriated Plena Code, were our efforts to introduce three-dimensional modeling and applications into water reactor safety. The effort was started sometime in maybe 1973 or 1974. We managed to push through a capital requisition for a basic 3-D solver software from Paul Nakayama and Jim Stuhmiller who were at SSS, a spinoff from SAIC. Buying any capital goods was not a straight-forward process. Justification for a software development group to buy software from somewhere else was very not straight-forward. We argued that development of such a code from scratch could not be accomplished at anywhere near the cost of the code.

The code was basically a bare-bones solver for the 3-D Navier-Stokes equations with a at-the-time standard two-parameter turbulence model included. There were no user-friendly aspects at all. But that’s all we needed because we were going to make a model that could be applied to 3-D analysis of the core region of power reactors. While the core region was the primary application focus, the downcomer, plena, and secondary side of the steam generator could also be handled in a three-dimensional manner.

We fully intended to use the code as the basis for development of models and methods for three-dimensional, transient, compressible, boiling two-phase flows of all kinds. A three-dimensional approach, for example, can be used to deeply investigate local physical phenomena and processes, and the results summarized for inclusion into lower-dimensional models and codes. A microscopic investigation of boiling processes at a heated wall would provide information for development of wall-heat partition functions; Hewitt (2016). We considered the potential application areas to be wide ranging. And now, four decades later, applications of two-phase 3-D CFD are finally underway in the nuclear industry.

In order adapt the code to the core region, we developed a three-dimensional mathematical model of fluid flow within the structured array of the fuel rods. The formulation could also be applied to the secondary side of steam generators. Irregular boundaries could be approximated by use of filled volumes representing regions where there was no fluid. These were basically no-slip boundary conditions, and could be used to represent solid structures anywhere in the solution domain. For the SPLEN plena code, the embedded solid structures would generally (but not universally) not be present, and the focus would be more toward turbulence.

A few reports and papers were based on the software including Wnek et al. (1975), Ramshaw and Trapp (1976), Hughes (1977), and Hughes and Chen (1977). The development was well along at the time that the new code development program at NRTS was shut down. In the mid- to late-1970s, research was underway on applications of three-dimensional thermal-hydraulics, including turbulence and eventually two-phase flows, to water reactor safety issues within the USNRC and DOE systems.

Citations to these very early 3-D CFD reports continue to the present time, especially whenever the concept of embedded solid structures are a part of the modelling. The French, in particular, have been early adapters of the concept and visited EI during one of their visits to Camp NRTS in the late 1970s. They had adapted the concept for applications to coal-fired, wet-wall boiler components. They could not understand why EI was not actively pursuing the code. We informed them that as a for-profit organization, we had to have a customer to pay for the work. Later we missed a chance to get significant EPRI funding when we lost a code run-off competition. EPRI supported 3-D model and code development for applications to steam generators for many years after we lost, and recently picked up the CFD approach for general applications.

The last code review committee meeting that I attended was in early 1975 in Washington, D. C. I was scheduled to present the status of our 3-D work and on my second slide I had words to the effect that we solve the 3-D Navier-Stokes equations. And I spoke those words, We solve the three dimensional Navier-Stokes equations. Someone in the audience, Clifford Truesdell if I correctly recall, said, in a rather loud and threating voice, You call those the Navier-Stokes equations!

The equation system that I presented was exactly the same as has been presented at thousands of technical meetings by thousands of engineers over several decades. At this point I resolved to make it through my presentation with the absolute minimum of interaction with the audience and in a dull, uninterested monotone.

Lance Agee was at the meeting as a representative of the Electric Power Research Institute. After NRTS management had presented the deadline ultimatum he had been successful in landing as a Research Project Manager at EPRI. At the break following my presentation Lance walked up to me and expressed complete dismay at what had transpired. He said that what he had just experienced was unbelievable and certainly no way to accomplish anything, much less make any long-range progress.

I looked at him and said, “I don’t care what they say, I’m leaving.” And I did. In July 1975. I had a job offer in hand, or maybe I had already accepted, to join a local private company in Idaho Falls, Energy Incorporated, (EI). The situation around our three-dimensional CFD work was the last straw for me. Continuing to work for the USNRC was absolutely not an option. It was certainly a waste of my time. Because of the adversarial environment it was a major waste of their money because they were throwing away many months of labor and years of experience and expertise.

I consider the environment fostered by the USNRC to be a mistake of the tenth magnitude. We had provided them with an opportunity to get a significant advance on the entire world and they blew it. Three-dimensional models, methods, and software applications for complex fluid flows is at present a robust and growing field. The present situation proves that our concepts from over 45 years ago were in fact the proper way to advance.

Bob Lyczkowski has documented the details of the factors that led to dissolution and dispersion of the NADA group. Very highly recommended. The History of Multiphase Computational Fluid Dynamics

3-D Codes References
W. J. Wnek, J. D. Ramshaw, E. D. Hughes, J. A. Trapp, and C. W. Solbrig, “Transient Three–Dimensional Thermal–Hydraulic Analysis of Nuclear Reactor Fuel Rod Arrays: General Equations and Numerical Scheme”, ANCR–1207, 1975.

E. D. Hughes, “Field Balance Equations for Two–Phase Flows in Porous Media”, in NSF Two–Phase Flow and Heat Transfer Symposium–Workshop, University of Miami, 1977.

E. D. Hughes and F. T. W. Chen, “Transient Three–dimensional Thermal–Hydraulic Analysis of Homogeneous Two–Phase Flows in Heat Exchangers”, AIChE Preprint Number 32, 17th National Heat Transfer Conference, 1977.

J. D. Ramshaw and J. A. Trapp, “A Numerical Technique for Low-Speed Homogeneous Two-Phase Flow with Sharp Interfaces,” J. Comput. Phys. 21, 438 (1976).