I note from Kash's post that he thinks that the price level will fall to 150% of the 1998 level, in real terms, by 2010. nl(1.5)/12 ~= .034, so by his estimates, housing will have a 3.4% real annual rate of return since 1998.
Since prices in 1998 weren't cheap, this still makes housing very expensive for people. "Rent for now" is a strategy that only works after 2002(ish.) And that isn't taking into account the tax benefits and the building of equity that people normally would do.
This kind of depresses me. I want to be able to afford a house. But it looks like I'll never get there, even at the end of the housing bust.
CR - thanks very much. In a few well-chosen charts Kash has highlighted some interested things. Worth everybody's time to ponder. My comment there:
Kash - thanks very much. Excellent breakdown of the market. Particularly appreciate YOY% changes illustrative power on prices. Also the macro-pattern that jumpbs out of coastal vs inland cities makes a great deal of sense but is not one that anybody seems to have pointed out. Great micro-principles though - coastal land being scarcer and in more demand. Equally interesting is the point about very long-run pricing adjustments in the real estate markets. With several well-chosen charts you've highlighted several key, heretofore not widely notice, structural patterns in the RE marketspaces. Even works against my experiences - i.e. Devenr was going thru a mini-Internet/Telecom boom in the late 90s and what used to be open space when I visited years ago was getting rapidly filled in. But now ?
It seems that this index has lost some credibility with econmists as of late and I don't think this newest report is going to help much - I have a hard time believing home prices were higher in Q4 than Q3 (ADP Home Price Report?).
Do you think the Case-Shiller index might be a viable alternative?
Depends on where you live ams... almost anyone can afford to buy a house in the town I live in now. Most folks quite easily & without 'exotics'.
Me too. In north central NE 100K will buy a very, very nice house with a few acres of land, of course the weather sucks but life has it's little trade offs.
Even Realtors out here have conceded that prices are down YOY here in LV, so what gives with OFHEO?
It is their measurement methodology, it tends to smooth & lag. It is based on transactions... something like:
(Current Price - Previous Price) divided by time between transactions.
I believe the differences are then adjusted so more recent data is weighted higher then older data... and also for inflation over the time period.
So say take two houses... each sells for $250k... house one was last sold for $50k twenty years ago (big gain), the other house previously sold for $255k just last year (slight loss).
1: $250k-50k/20 = $10k/year... then adjust for 20 years inflation & weight
2: $250-255k/1 = -$5k/year... then adjust for 1 yr inflation & weight
Sum these & all others & you get a pretty smooth & lagged curve compared to what is probably actually happening in the market at that time.
And it is probably skewed a little toward 'positive' since we've had so much more gain lately than decline that even if weighted toward 'more recent data' the past run up just overwhelms the data.
I am not EXACTLY sure that is how they do it but I understand it is something like that and that is why it misses the turns (index lags & dampens).
After the bottom is hit OFHEO will probably miss the return to appreciation too - same reasons but in reverse.
Sorry to tell you this ams16 but if you can't afford around a 3% increase in prices, then you have not been keeping up with inflation. It has nothing to do with buying a house but you making yourself more valuable or getting wage increases to keep up with inflation.
Problem is, there's probably a good reason for that, like maybe there aren't good paying jobs in tech...
Not many but some.
One of my son's friends graduated a couple years ago (IT-math double major very good grades & internship)... looked out wet, looked out east... ended up working for a small software development company making applications for agri-business (ethanol plants, co-ops, etc).
Paid about the same as what he could get on either coast (maybe 10% less locally - but close though).
The difference is the cost of living. If he had gone to Cali he would never own a house... But in fly over he is two years out of college & he and his girl are looking at buying a house... it will likely cost less than their combined annual incomes. Yes - very cheap, see Kash's chart on 'inland cities'... they started cheaper & didn't appreciate much.
But this town isn't real 'exciting' either... you have to make some choices. My guess is they will be making babies here pretty quick & won't be havin' much excitement other than tours at midnight feedings.
Not sayin' everyone should do this. People make choices - what works for some aren't right for others.
I forgot the gov't is in the business of taking a simple formula and construing, complicating, and confusing anyone trying to figure out how the gov't came up with their numbers so they don't even try and just give up and believe what it is they are being told.
Hey Dryfly, My guess is, almost everyone in the country can afford to buy a house in CA! No doc, no down option adjusted loans are widely advertised. We're inundated with 3-5 tv infomercials DAILY telling all about opportunities to fi or refi. Even the Donald makes trips out here to advise us all of the opportunities in r.e. (Although I did hear the seminar cost was reduced from $150 to $0. or so.)
It seems the only absolute qualification one must meet here is he'she MUST BE STUPID!
Depends on where you live ams... almost anyone can afford to buy a house in the town I live in now. Most folks quite easily & without 'exotics'.
I'm stuck in the Boston area because I'm tied to university life and because of family.
HotKarlRove: Note I said REAL increase, not nominal. Kash's graphs are in REAL terms, discounting inflation. Please read a post before disparaging the poster.
I am in a precarious position. I have been saving for the past 5years for a down payment to buy a SFH in the Northern Virginia suburbs. I had planned to purchase one this spring and move in. I felt the market was slow and it was a good time to buy.
One day while browsing the web I hit CR and now I am freaked out as to what to do. From my research on the neighborhood and the location everything is a go. The only thing that bothers me is the prices are so high if I buy a place and the market does collapse I am in deep doo doo. If the prices go up I am in deep doo doo.
After reading up several articles on this site I have no clue what to do next. On the other hand the wife keeps nagging away on buying a house. if u were in my boat what would u folks do???
Masking tape on the nest-builder (wife) pronto. Tie her to the kitchen chair and then show her graphs of recent price changes and inventory in your area.
Your final compelling message: Wait for sping after next spring.
The only thing that bothers me is the prices are so high if I buy a place and the market does collapse I am in deep doo doo. If the prices go up I am in deep doo doo.
You are only in deep doo-doo if you can't make the payments.
So what if it goes go down in price, if you aren't selling soon for a profit & you can make the payments at the price you bought & you like the place - what's the problem?
On the other hand if it does go up more & you make enough so you can make the payments then - what's the problem?
The reason all these people got into trouble is they bought homes they can't make payments on them & live too - it was not sustainable over the long haul.
That is the issue to focus on - not so much the market ups-s-downs... focus on your personal economics!
The only thing that bothers me is the prices are so high if I buy a place and the market does collapse I am in deep doo doo.
If I were you, Mulik, I'd ask myself why you think this.
If what you wish to buy is a home to live in, and you can comfortably afford the payments (plus maintenance and the transaction costs and all that), and you intend to live there for quite some time, and think that intention is reasonable given your current employment and the likelihood you could get a different job of equal earning potential without moving, if you had to, then why are you in deep doo doo if prices fall after you buy?
If the only way you can afford to buy is to stretch yourself too far, or if your situation (employment, family, whatever) means that you might very well need to move in two years, then certainly you should be cautious about buying in a deflating market.
My perspective comes from someone who has been a mortgage lender, not from someone who has ever been a real estate investor. So I tend to think of homes as homes, not as "investments," and loans as ways to buy homes. I tend to think of the "investment" part, if it happens, as gravy. Just about anyone in this country who has owned a home for ten years or more has experienced periods where that home depreciated or at least didn't appreciate. It still remained a home in that time, and as long as the mortgage payments weren't crippling, most people didn't get that worked up over it. They did, after all, still have a home.
I guess this all reminds me of locking in interest rates on a loan application, which is something I do have a lot of experience dealing with from the advice-dispensing side of things. If you consider yourself risk averse--if the thought of what you lose with a higher rate bothers you more than the thought of what you gain with a lower rate--then lock the loan. If you really think rates might go down, or you're really bothered by the possibility that you'll miss a dip, then float the loan. But don't ask your lender what to do, because your lender doesn't have the right answer for you. The lender's perspective is not yours.
The OFHEO looks like they're calculating their data the way I do, using apples-to-apples comparisons of current calendar month to the same calendar month the previous year. Less noisy, less volatile, fewer bad "signals."
Not as dramatic a story, of course, just a more accurate reflection of what's going on.
Thanks for your response and your opinions. I guess I have to rethink this.
The doo doo on the downside came in because I had put so much effort to save all that money over the past 5yrs. This is honest hard earned money. It felt like I might be gambling my money away if prices went down. I have lived way below my lifestyle to save up the money.
The doo doo on the upside came if I didnt buy one now and had to save some more to make the down payment.
Again I think the fault is mine being a engineer I had never tried to follow up on what effects interest rates had on assets. Being out of college recently didnt help either.
Forgot to mention. here in northern virginia there is hardly any place less than 600K which is in a good shape. Even though I make good money I am still a alt A candidate.
I buy a place and the market does collapse I am in deep doo doo. If the prices go up I am in deep doo doo.
Mulik, this will likely be one of the more important joint decisions of your life. Some things to consider:
1. Is it truly a joint decision?
2. Have you both educated yourselves about the rent verses buy optionas available to you?
3. Have you both carefully considered your commitment to Northern Virginia?
4. Have you both carefully studied the housing market?
5. Can you qualify for a house and comfortably afford the payments? I foolishly listened to a real estate agent for our first home purchase and was house poor for several years (and divorced shortly after the purchase: did not listen to my wife).
6. Will you be able to get a "prime" comforming loan? If you need an interest only loan or an ARM, then maybe it is not time to buy.
7. Will you be able to rent the house for something close to your payment if you have to move and cannot sell?
8. Are you following the DC area housing blogs? One of my coworkers was considering purchasing an invesment condo in the Northern D.C. suburbs (his wife wanted it). One of the blogs talked about the exact complex they were considering. After doing some research, they are going to wait and see what happens. She is a CPA.
There is no hurry, prices are not likely to move aggressively higher in the next few months.
Again I think the fault is mine being a engineer I had never tried to follow up on what effects interest rates had on assets. Being out of college recently didnt help either.
I'm an engineer too and I learned something about this stuff - though not enough to be confused as an underwriter...
You have time.
Rob'ts points (above) are very good. Focus on Point #5. Payments over the long haul. That is the key.
That and how long will you plan to stay there. If you think you will be moving fairly soon (like within 3-5 years) then save more & do not buy. Rent & rent wisely (look for value - it is out there).
If you think you will be there a long time, then the ups-n-downs wash out. Then it doesn't really matter if you pay a little too much when you buy.
As long as you can still make the payments that is. Point #5.
Some folks on this post are asking why the OFHEO data is not showing a drop in prices. CR, didnt you once mention that the OFHEO summaries are based only on the GSE data ? Given that the GSEs dabble mostly in prime, does this explain at least part of the discrepancy ? Still learning
dryfly - how many times do you want me to debunk this "example" you keep posting? OFHEO's methodology is explained on their website universe
NJ-Bob - I believe you are on to something, and if we're right, the uptick in the ABX was exactly the wrong response to the OFHEO release. The Case Shiller index uses more or less the same methodology, but with different data. The small methodological differences might explain some it, but a) they are pretty arcane and b) I doubt it. So although I'll leave open the possibility that it's methodology, I'll bet it's data.
One data difference is timing. CS uses recorded transactions, which can lag months but usually only lag a month or so (which is why December is released in late Feb.) OFHEO uses loans bought by the GSEs, which often lag by a few months. So the first release of the OFHEO index (later revised) is weighted towards the beginning of the quarter (more Oct. loans have made it in than Dec. loans). True for CS, but to a much smaller extent. But CS shows a drop for every month of the quarter, so that can't explain more than a part of it.
Another is the fact that it's GSE data, so the national weighting is different. California especially would have less weight for OFHEO than it would for CS. But you can look at the OFHEO indices for San Diego or SF and compare them, and CS is substantially more negative. Unfortunately, you can't do a perfect job on that since OFHEO doesn't release the purchase only at the MSA level, and you have to make the wild guess that the appraisal bias doesn't differ much between markets. Still, it doesn't look like the different weighting for California explains all of it.
The third difference is that OFHEO has no jumbo loans, while CS does. If expensive houses were falling faster (in percentage terms) then CS might drop while OFHEO rises. But if you look at cheap MSAs where there aren't many jumbos (Detroit or Chicago) OFHEO and CS don't get much closer (again with the warning that the comparison isn't quite right because of the purchase-refi issue).
But the last difference (you knew there was a reason you kept reading. you did keep reading, didn't you?) is that OFHEO would have very few subprimes, while CS would have almost all of them. If prices in subprime heavy neighborhoods are falling faster than prices in other neighborhoods (makes sense if subprime originations are dropping fast, it's where the demand would be disappearing the fastest) you'd get what we see, a big divergence between OFHEO and CS. And that should scare the bejeepers out of anyone long the ABX.
I buy a place and the market does collapse I am in deep doo doo. If the prices go up I am in deep doo doo
The doo doo on the downside came in because I had put so much effort to save all that money over the past 5yrs. This is honest hard earned money. It felt like I might be gambling my money away if prices went down. I have lived way below my lifestyle to save up the money.
I'm jack to you, so you might consider my opinion with that in mind.
You seem to have pretty much answered the question yourself. You are considering buying a house as an investment, in addition to a place to live.
I would say you think that through - because you need to invest for your future needs(retirement, kids education etc), and you need a place to live.
If your house as an investment does not pan out, can you meet your other future needs?
Just find out what you can afford to spend for housing alone, not housing & investment. If you can afford to buy on that, buy. If you cannot, rent. That "afford the payments", for me is what I can afford for shelter, with a reasonable margin for little more for owning.
Historically, house prices have just kept pace with inflation and income, long term. So prices cant go up forever disconnected to income, and prices will regress to what incomes can support. That can happen due to rising income, inflation, combination of both, whatever. So "buy now, or be priced out forever" is unfounded.
If you live in a place where the people who own, cannot afford to buy now, rest assured that prices and income will adjust, or businesses and jobs will move out.
First paragraph of the Introduction from the link you provided...
"The house price
indexes published by OFHEO -- hereafter referred to collectively or individually as the HPI -- are based on a modified version of the weighted-repeat sales (WRS) methodology proposed by
Case and Shiller (1989)."
What do you think 'weighted' & 'repeat sales' mean?
But read a little farther...
"The repeat sales method was first proposed by Bailey, Muth, and Nourse (1963), and later
extended by Case and Shiller (1987, 1989).9 This approach limits the extent to which changes in the
composition of the sample used for estimation can influence the estimated index. Utilizing information
on the values of the same physical units at two points in time controls for differences in housing
attributes across properties in the sample without directly estimating their marginal contribution to total
value. A multivariate regression is employed to account for the fact that all properties do not transact
in every period. The repeat sales method is the only approach that provides an opportunity to develop
constant quality house price indexes using GSE data. The lack of information on detailed property
characteristics in historical GSE data precludes the estimation of hedonic house price indexes."
Pretty much says it all - good link you had there.
The rest of the paper is basically how they weight and why...
But as I said SOMEHOW they track actual sales of actual homes - over time - as opposed to just summing up sales results at large per region per time bucket & comparing with a similar sample later in time.
To avoid the problem of different time scales & mix effects... they use the sampling & weighting methodology described in the paper. Not sure I understand why they do what they do but believe them that it is done.
I would say that "repeat sale" means they use prices for the same property that sell repeatedly. And I would say that "weighted" means that they give different weights to different observations. Would there be some other meanings?
I would say that "repeat sale" means they use prices for the same property that sell repeatedly.
But the paper doesn't say its the same properties over and over... just that they track repeat sales over time.
It does say that they correct for the various time differences... in fact they say that is part of the strength of the method in that it bridges various periods of time & corrects for sales mix. Suggest to me it isn't just the same homes repeatedly selling.
This is in part why the index damps & smooths and miss a hard change in market - either up or down.
I don't have any idea how many properties they use in their tracking samples... nor how often they use the same ones over (if they do - I assume they do if homes they are tracking or have previous data on happen to sell in the period of interest)... and I don't understand how they weight or how they then extrapolate the results of this sampling to the whole population... but I know they do & that is sufficient for me.
dryfly - why exactly would bridging various periods of time and correcting for sales mix result in damping and missing turning points?
Aren't those exactly the things that you want an index to do?
Aren't those exactly the things that you want an index to do?
Yes - IF they could get an instantaneous 'snapshot' without the sales mix error. But they can't. if they go with a 'snapshot' the noise hides the changes too but instead of not reporting them it 'over reports' in a series of false negatives.
So they use some kind of 'transaction tracking' algorithm to reduce the sales mix variation... but by doing that they dampen the index's response to rapid changes or inflections... in effect because of the bridging.
And if they ONLY used homes that sold rapidly & often... that too would skew the sample mix. Houses that sell often, sell often for a reason... it isn't random... they need randomness & a large tracking sample size to make the thing work.
In most cases (most periods) bridging would 'help' (or at least not hurt) because most of the time price curves of big lumbering markets like this are pretty well behaved & smooth to begin with so having multiple durations (weighted somehow) would tend to reduce noise (reduce sample error).
But it will make it much harder to pick up an abrupt REAL change in the signal one way or another because you bridge over the spike or dip. Eventually it shows up... but lagged & muted.
I don't pretend to know all about how they do it... but I gather from the piece you posted & other stuff I've read elsewhere... that their strategy is...
(1) track actual transactions over time for a sample population they have data on (the repeat sales)
(2) adjust & weight results from transaction tracking for sample mix & duration variations (the weighting)
(3) somehow project weighted sample results over entire 'universe' of housing stock to generate the index
I'd love to know more, the article doesn't explain the 'tactics'... just the 'strategy'.
And obviously I might be wrong - but that is how I read it.
here is a link to mid altantic OFHEO price appreciation vs inflation. basically this gives a person an excellent guide when to buy RE relative to the rate of inflation.
dryfly - your first paragraph doesn't make any sense, your second paragraph simply repeats your assertion, your third paragraph starts with a hypothetical that they don't do, and your fourth, fifth, and sixth paragraphs just repeat another assertion.
The paper does provide the tactics as well as the strategy. They run a regression with the dependent variable as the log of the sales price, and the independent variables are a series of indicator variables that are set to -1 in the quarter of the first sale, 1 in the quarter of the last sale, and 0 for all other quarters. That's right there in the paper.
Why is the HPI based on Fannie Mae or Freddie Mac mortgages?
OFHEO has access to this information by virtue of its role as the federal regulator responsible for ensuring the financial safety and soundness of these government-sponsored enterprises. Chartered by Congress for the purpose of creating a reliable supply of mortgage funds for homebuyers, Fannie Mae and Freddie Mac are the largest mortgage finance institutions in the United States. Their combined mortgage records form the nation's largest database of mortgage transactions.
the detailed description of the data is on p. 4 of the above link to the research paper, and the description of the regression used to generate the index is on p. 7
Right... They have a big pool, thats why they use regression.
Then the the second question is are the quarters (the -1,0,1's) all in the same quarters? If NOT in the same quarter then that's your 'bridging' right there...
In effect it turns into a very large vector 'matrix' problem.
The regression is just the way they process gazillions of transactions spread out all over time... They have a huge sample... I just used two in my 'simple example' - one long, one short.
I doubt they use the same homes over & over again in their pool... waiting for them to 'resell'. Instead they probably mine the GSE databases for transactions and then have some kind of selection criteria to randomize.
That is your Step 1 in my post above.
That still doesn't tell me how they weight for time or mix... but they do, says so.
"Then the the second question is are the quarters (the -1,0,1's) all in the same quarters? If NOT in the same quarter then that's your 'bridging' right there..."
you know, I can't even begin to parse that paragraph
I am watching NoVa as well, Loudoun. I think you probably focus on Fairfax. But anyway,as Tanta and Dryfly said, a home is a home, a place to live and raise kids. As to the market in NoVa, here is a blog. Blogger updates local market everyday. Good information.
you know, I can't even begin to parse that paragraph
mort - they are building a matrix of transactions... from GSE databases. -1,0,1s... in fields for the quarters for which previous purchase, no purchase & current purchases (respectively) occur. Right?
Then fit via regression.
So are all the 'previous purchases dates... the occurances of those (-1)s always in the same quarter? Or are they spread out into the past for different homes & different transactions?
Example: we are in 1Q07... say we want OFHEO for 4Q06 (quarter just passed)... do we ONLY look at transaction with previous sales in one quarter (say arbitrarily, 4Q05)?
That would result in a matrix with all the -1's in 4Q05, all the 1's in 4Q06, and zeros everywhere else. Matrix would look like this:
Past qrts to Current qtr
0...0(-1)000:1
0...0(-1)000:1
0...0(-1)000:1
So on through the set of all transactions in the matrix...
Or do they use multiple transaction durations in the sample set (samples where the previous purchases could have occurred at many times in the past)... then the matrix would look something like this:
000(-1)00:1
0(-1)0000:1
0000(-1)0:1
(-1)00000:1
With the above example only looking back six quarters... could look back lots farther.
If the transaction times of the previous purchase in the sample set are spread out over time then you will have bridging and as a result smoothing.
If not - if they have a frozen duration period (and keep it real short) they will have much less bridging & less smoothing but will then have sample bias error (by limiting their sample set to only those house that flip often).
From reading the article - and watching how slowly OFHEO index moves - my guess is they bridge quite a lot.
again, the creation of the data is described on p. 4. As the paper indicates, and as I've indicated in past threads, they have multiple durations. they use all transaction pairs in data that goes back to 1975.
If by "smoothing" you mean they only produce 1 number per quarter, then how would you produce an index any other way? If by "smoothing" you mean that the amplitude of the index is consistently less than reality, then why would "bridging" as you call it produce "smoothing." You've repeatedly asserted this, but never claimed a reason.
The quarterly index has gone from around 4% per quarter to about 0.5% per quarter in about 2 years. That's slow? I'd hate to see fast. Do you honestly believe that housing was going up substantially faster than 4% per quarter as a US average 2 years ago, and that the index substantially understated it??????
let me modify that slightly. you've never claimed a reason consistent with the index being created via a regression on log prices. back when you were stating that they took some sort of average over durations you almost got to the point of claiming that averaging these averages produced the smoothing, but now that we're past that, what mechanism do you assert causes this "smoothing."
Hey Mulik, I forgot to mention that we have software jobs here in Omaha. System integrators: Weblogic (with Plumtree), Oracle, Websphere. J2EE programming is a plus. Security clearance a plus but not critical as long as you can get one eventually. Salaries are good because folks don't like to move here. Technical market is solid (insurance, Cargil, Union Pacific, DoD). Schools are great. Housing is inexpensive. I've lived and worked in CA, WA, and CO and, other than the weather, I like Omaha the best. I walk to work from a brand new house. We chickened out on building a large house on the executive lot across the street from the plant: for $400K you can buy our lot and build a McMansion. For a lot less you can buy a nice starter house. We will not likely retire to Omaha since our family is on the West coast, but it is a great place to work and reaise a family. When I was in the service, I spent two training tours at Ft. Belvior- I really like NoVa, but there are sooo many people now and it is soooo expensive.
but now that we're past that, what mechanism do you assert causes this "smoothing."
The bridging causes the smoothing.
Take a look at my early example... Say there are two houses selling right now... both for $250K (just a convenient coincidence).
House #1 was previously bought 20 years ago for $50K then sold this year for $250K... Net gain.
House #2 was previously bought last year for $255K and sold this year for $250K... Net loss.
1: $250k-50k/20 = $10k/year.
2: $250-255k/1 = -$5k/year.
What does that tell us about housing prices at this very moment? Does it tell us houses have gone up or houses have gone down?
If you don't like using houses like House #1 in the dataset and only use houses like House #2 because #2's previous purchase was more recent, then you have made a selection bias right from the get go against houses with ownership stability.
If you include House #1 in the data set... then you have 'bridging problems'... because maybe House #1 went down in value last year too, but it is unknown since there was no transaction to uncover it.
If you use a lot of homes like House #1 in a large data set you will smooth out the response because of the hidden data that never gets captured. In that 20 year period prices went up and down quite a lot but house #1 missed it all.
If you throw out all homes like House #1 and only look at recent transactions you get a skewed sample for another reason BECAUSE houses like House #2 that sell often are not necessarily representative of the entire universe of all houses. We all have homes in our neighborhoods that sell often - the reason isn't always 'random'.
I believe OFHEO includes some if not a lot of homes like Home #1 - though not necessarily with twenty year old transaction histories - as well as those with the most recent transaction histories. I believe they use a staggered mix to try and reduce the sample bias error.
I believe they try and reduce the bridging errors by weighting transactions histories differently with those with more recent transactions histories higher.
That is how I interpret 'weighted repeat sales' from the article.
In short - if they reach back into the past to multiple start times they will have smoothing (signal dampening). If only because of the lost pricing information they passed over.
If they only use short selling periods (all recent) they will have sample bias & noise.
I believe OFHEO tries to split the difference between these two risks and I believe they do a good job. I don't know a better way to get a meaningful 'global number' but believe there are still issues with the index nonetheless... the biggest being the smoothing (or signal dampening).
Interesting discussion of the innards of the OFHEO index, but I still can't get past the fact that it is not really a sales index, but is rather a mixture of sales and appraisals for refinancing. In today's market, it's quite easy for me to find an appraiser to tell me my house is worth, say $300k, but quite impossible to find a real live buyer who would pay that. Unsurprisingly, the purchase-only index is lower than the combined index, which is the headline number. If you take the ratio of the HPI index annual change to the purchase only index on the spreadsheet you can download from OFHEO, the HPI index starts to diverge from purchase only in 2004 3Q. The ratio has gone from 1.03 in 20042Q to 1.43 in 20064Q, indicating that appraisals are less and less anchored to reality, and that the headline HPI is less meaningful.
szara - the MSM only talks about the purchase with appraisal refi index, but OFHEO also publishes on their website and includes in their press release the purchase only index. I tend to ignore the index discussed in the MSM and look only at the purchase only index.
dryfly - you're welcome to believe what you want about how OFHEO calculates its index, but the document on their website that I posted the link for contradicts much of what you seem to believe. But you are welcome to believe it anyway.
Since you've reposted your example, I'll repost my counterexample
;
dryfly - if they had only one observation you'd be right, there'd be no way to know the change between year 19 and year 20. But let's say they had two observations, one that goes up 220% in 20 years, and one that goes up 245% in 19 years. It is then the proverbial piece of cake to figure out that the longer observation represents a fall of about 10% in the 20th year. And they actually have more than two observations. They have millions of observations spanning various parts of the 20 year period. Parceling out that 20 year gain between the first 19 and the last one is a fairly trivial excercise in minimizing the sum of squared residuals.
mort_fin | 02.26.07 - 10:19 pm
mort your 'contradiction' doesn't address any of the signal dampening issues I mention. Consider:
if they had only one observation you'd be right, there'd be no way to know the change between year 19 and year 20. But let's say they had two observations, one that goes up 220% in 20 years, and one that goes up 245% in 19 years. It is then the proverbial piece of cake to figure out that the longer observation represents a fall of about 10% in the 20th year. And they actually have more than two observations. They have millions of observations spanning various parts of the 20 year period.
Your 'contradiction' still doesn't address the smoothing due to different transaction durations for different units even if 'millions of them' are included.
Remember they track same unit repeat sales over different time periods - to avoid the issue of sales mix. So what happens to another house isn't necessarily reflective in what happens to THIS house. Else they would just use a 'global' sum & mean.
Think of each of these million transactions as a separate line each with its own 'slope'... sum all the millions and you get:
With each Xi being a different house, each line a different transaction and each tij a different time j (2 being recent & 1 being previous) for each home i.
It doesn't 'look' like this because they use the -1,0,1 format but is the same math... an array of separate transactions.
From a geometric perspective, the regression is fitting all those lines to one curve... the OFHEO index. It is quite damped.
in the example that I gave the index isn't "smooth." It rises for 19 years, and tanks in 1.
You keep asserting that the index is fitting slopes, and that this causes smoothing. But you're presenting no evidence that the index is fitting slopes, and the paper describing the creation of the index explicitly states that they are fitting endpoints - all the independent variables between the dates of sale are zero. For a property that rises 21% over two years, a pattern of 10%, 10% does not fit the observation any better than does a pattern of 21%, 0%, or 0%, 21%, or for that matter, -10%, 34% or 34%, -10% (give me a few basis points for rounding). If observations on sales that occur in between this property's sales indicate that a pattern other than 10%, 10% fits better, than their is nothing in the fitting of this observation to bring the result back towards 10%, 10%.
But you're presenting no evidence that the index is fitting slopes, and the paper describing the creation of the index explicitly states that they are fitting endpoints
Yes... but the end points 'connect' a transaction history... end point 'now' minus end point 'then'. That results in a slope effect over the intervening period for THAT transaction history. Sum & fit a big number of them and you get a smoothed & dampened curve. No way to get around it using this algorithm.
Now if all they did is look at endpoint 'now' and exclude all previous 'thens'... then look at endpoint 'then' and exclude 'now', that would result in little more than two global sums & averages.
The sum (& distribution) of {Xi(t2i)-X(t1i)} can be very different from the sum (& distribution) of {Xi(t2i)} minus the sum (& distribution) of {Xi(t1i)}... The first is a vector process with slope the latter a scalar process.
I believe because they are tracking repeat sales of the same units the process is far more like the former than the latter...
Oh and the 'evidence' of this is in the regression fitting of the (-1,0,1) matrix... that is in effect the equivalent of fitting the vector array made up of all {Xi(ti2)-Xi(ti1)} with the Xi(ti2)'s corresponding to the 1's and the Xi(ti1)'s corresponding to the -1's... the 0's all the intervals before & between.
"The sum (& distribution) of {Xi(t2i)-X(t1i)} can be very different from the sum (& distribution) of {Xi(t2i)} minus the sum (& distribution) of {Xi(t1i)}... "
It is algebraically impossible for the sum of {Xi(t2i)-X(t1i)} to be different from the sum of{Xi(t2i)} minus the sum of {Xi(t1i), as you assert.
And they are not summing endpoints. They are minimizing the sum of squared residuals between actual endpoints and endpoints predicted by a set of index values. That's what a regression is.
Say you have 3 transactions. One goes up 20% between year 1 and year 2. The second goes down 24% between year 2 and year 3. The third changes by 0% between year 1 and year 3. Explain to me a) why the index wouldn't register a 20% increase for year 1 and a 24% decrease for year 2 or b) why those index values would represent smoothing.
It is algebraically impossible for the sum of {Xi(t2i)-X(t1i)} to be different from the sum of{Xi(t2i)} minus the sum of {Xi(t1i), as you assert.
I have to think about that one - it wasn't a good example. I know you are right if X is linear & there are no sample selection issues for X(t) period to period.
But if X(t) is nonlinear, or if there are sample selection issues creating different samples period to period, then I'm not so sure.
The price/valuation curve OFHEO is trying to map probably isn't linear.
And there are sample selection issues (the units that sell vs. those that don't sell spread out over different intervals).
Regardless it was still a bad point on my part.
I know you know a lot about regression & statistics in general... can see that. But I think you are missing or underestimating the impact from the hidden appreciation of the previously unsold units carried forward that don't show up until that particular unit is sold.
Until a transaction takes place that appreciation is 'zeroed' out in the regression.
Realize I am not interested in the OFHEO index vs some other index but rather OFHEO vs what is really happening in the actual real estate market & does OFHEO always measure that well. That is what you want an index to do - track reality.
I think it lags & is dampened compared to what is happening in the 'real world' and I think we are seeing that now with OFHEO still increasing but real markets probably already heading down.
Even though I don't think it is a terrible index, I think OFHEO has missed this turning point. I believe it will also miss the initial change toward a future rebound - whenever that happens.
But I clearly don't have a complete enough grasp of the mechanics to explain why I 'feel' this way. I have to read more & crunch some numbers before I will be able to do that.
Have you any other references besides this paper - ones where they actually walk through the calculation with data sets?
one or more of those three might help. Also, if you have access to academic searches, an article by Brad Case (not related to Karl Case) and Henry Pollakowski in the mid 1990's in, I think, the American REal Estate and Urben Econ Jrnl (but I might have that wrong) walked through a lot of these issues.
And properties aren't "zeroed" until they are sold. They are not included in the regression at all until they sell.
So who will be left holding the bag ?
This is was a good time to find some long term place to mix some foreign currencies etc..
So I found this fund, which sounds nice "Global government bond fund" and I was thinking of investing in it.
At times like this one should read the prospectus.
here is the answer (1 among many) to the question: Who will be left holding the bag.
Allianz Global Investors: Closed End Funds | About this Fund
the answer starts at page 12.
I wonder where else are all those MBS found their way into ????
I note from Kash's post that he thinks that the price level will fall to 150% of the 1998 level, in real terms, by 2010. nl(1.5)/12 ~= .034, so by his estimates, housing will have a 3.4% real annual rate of return since 1998.
Since prices in 1998 weren't cheap, this still makes housing very expensive for people. "Rent for now" is a strategy that only works after 2002(ish.) And that isn't taking into account the tax benefits and the building of equity that people normally would do.
This kind of depresses me. I want to be able to afford a house. But it looks like I'll never get there, even at the end of the housing bust.
NEW layoffs
I'm hearing there were big layoffs in San Diego office today.
CR - thanks very much. In a few well-chosen charts Kash has highlighted some interested things. Worth everybody's time to ponder. My comment there:
Kash - thanks very much. Excellent breakdown of the market. Particularly appreciate YOY% changes illustrative power on prices. Also the macro-pattern that jumpbs out of coastal vs inland cities makes a great deal of sense but is not one that anybody seems to have pointed out. Great micro-principles though - coastal land being scarcer and in more demand. Equally interesting is the point about very long-run pricing adjustments in the real estate markets. With several well-chosen charts you've highlighted several key, heretofore not widely notice, structural patterns in the RE marketspaces. Even works against my experiences - i.e. Devenr was going thru a mini-Internet/Telecom boom in the late 90s and what used to be open space when I visited years ago was getting rapidly filled in. But now ?
CR,
It seems that this index has lost some credibility with econmists as of late and I don't think this newest report is going to help much - I have a hard time believing home prices were higher in Q4 than Q3 (ADP Home Price Report?).
Do you think the Case-Shiller index might be a viable alternative?
I don't get it.
Are house prices moving up by %5.9 a year or down ?
who are we to trust ?????
"I have a hard time believing home prices were higher in Q4 than Q3"
Even Realtors out here have conceded that prices are down YOY here in LV, so what gives with OFHEO?
Since they use apples to apples comparisons, could inflated appraisals from refi's be bumping the numbers?
This kind of depresses me. I want to be able to afford a house. But it looks like I'll never get there, even at the end of the housing bust.
Depends on where you live ams... almost anyone can afford to buy a house in the town I live in now. Most folks quite easily & without 'exotics'.
But it isn't coastal California or Manhattan.
just heard on CNBC that the oracle Buffet says a soft landing is wishful thinking and we could get political backlash
dryfly
Depends on where you live ams... almost anyone can afford to buy a house in the town I live in now. Most folks quite easily & without 'exotics'.
Me too. In north central NE 100K will buy a very, very nice house with a few acres of land, of course the weather sucks but life has it's little trade offs.
Depends on where you live ams... almost anyone can afford to buy a house in the town I live in now. Most folks quite easily & without 'exotics'.
Problem is, there's probably a good reason for that, like maybe there aren't good paying jobs in tech...
Even Realtors out here have conceded that prices are down YOY here in LV, so what gives with OFHEO?
It is their measurement methodology, it tends to smooth & lag. It is based on transactions... something like:
(Current Price - Previous Price) divided by time between transactions.
I believe the differences are then adjusted so more recent data is weighted higher then older data... and also for inflation over the time period.
So say take two houses... each sells for $250k... house one was last sold for $50k twenty years ago (big gain), the other house previously sold for $255k just last year (slight loss).
1: $250k-50k/20 = $10k/year... then adjust for 20 years inflation & weight
2: $250-255k/1 = -$5k/year... then adjust for 1 yr inflation & weight
Sum these & all others & you get a pretty smooth & lagged curve compared to what is probably actually happening in the market at that time.
And it is probably skewed a little toward 'positive' since we've had so much more gain lately than decline that even if weighted toward 'more recent data' the past run up just overwhelms the data.
I am not EXACTLY sure that is how they do it but I understand it is something like that and that is why it misses the turns (index lags & dampens).
After the bottom is hit OFHEO will probably miss the return to appreciation too - same reasons but in reverse.
Sorry to tell you this ams16 but if you can't afford around a 3% increase in prices, then you have not been keeping up with inflation. It has nothing to do with buying a house but you making yourself more valuable or getting wage increases to keep up with inflation.
Problem is, there's probably a good reason for that, like maybe there aren't good paying jobs in tech...
Not many but some.
One of my son's friends graduated a couple years ago (IT-math double major very good grades & internship)... looked out wet, looked out east... ended up working for a small software development company making applications for agri-business (ethanol plants, co-ops, etc).
Paid about the same as what he could get on either coast (maybe 10% less locally - but close though).
The difference is the cost of living. If he had gone to Cali he would never own a house... But in fly over he is two years out of college & he and his girl are looking at buying a house... it will likely cost less than their combined annual incomes. Yes - very cheap, see Kash's chart on 'inland cities'... they started cheaper & didn't appreciate much.
But this town isn't real 'exciting' either... you have to make some choices. My guess is they will be making babies here pretty quick & won't be havin' much excitement other than tours at midnight feedings.
Not sayin' everyone should do this. People make choices - what works for some aren't right for others.
Thanx dryfly
I forgot the gov't is in the business of taking a simple formula and construing, complicating, and confusing anyone trying to figure out how the gov't came up with their numbers so they don't even try and just give up and believe what it is they are being told.
my bad
Hey Dryfly, My guess is, almost everyone in the country can afford to buy a house in CA! No doc, no down option adjusted loans are widely advertised. We're inundated with 3-5 tv infomercials DAILY telling all about opportunities to fi or refi. Even the Donald makes trips out here to advise us all of the opportunities in r.e. (Although I did hear the seminar cost was reduced from $150 to $0. or so.)
It seems the only absolute qualification one must meet here is he'she MUST BE STUPID!
Depends on where you live ams... almost anyone can afford to buy a house in the town I live in now. Most folks quite easily & without 'exotics'.
I'm stuck in the Boston area because I'm tied to university life and because of family.
HotKarlRove: Note I said REAL increase, not nominal. Kash's graphs are in REAL terms, discounting inflation. Please read a post before disparaging the poster.
I'm stuck in the Boston area because I'm tied to university life and because of family.
Well I can understand the family thing... but believe it or not there is a college or two west of the Berkshires.
but believe it or not there is a college or two west of the Berkshires.
Yes, but tenure track positions don't grow on trees.
Question to CR, Tanta and the rest
I am in a precarious position. I have been saving for the past 5years for a down payment to buy a SFH in the Northern Virginia suburbs. I had planned to purchase one this spring and move in. I felt the market was slow and it was a good time to buy.
One day while browsing the web I hit CR and now I am freaked out as to what to do. From my research on the neighborhood and the location everything is a go. The only thing that bothers me is the prices are so high if I buy a place and the market does collapse I am in deep doo doo. If the prices go up I am in deep doo doo.
After reading up several articles on this site I have no clue what to do next. On the other hand the wife keeps nagging away on buying a house. if u were in my boat what would u folks do???
Any suggestion is appreciated.
Yes, but tenure track positions don't grow on trees.
You can say THAT again. Who needs to own a home if you can get tenure. In that case 'rent' trumps.
Seriously - not snark.
Masking tape on the nest-builder (wife) pronto. Tie her to the kitchen chair and then show her graphs of recent price changes and inventory in your area.
Your final compelling message: Wait for sping after next spring.
The only thing that bothers me is the prices are so high if I buy a place and the market does collapse I am in deep doo doo. If the prices go up I am in deep doo doo.
You are only in deep doo-doo if you can't make the payments.
So what if it goes go down in price, if you aren't selling soon for a profit & you can make the payments at the price you bought & you like the place - what's the problem?
On the other hand if it does go up more & you make enough so you can make the payments then - what's the problem?
The reason all these people got into trouble is they bought homes they can't make payments on them & live too - it was not sustainable over the long haul.
That is the issue to focus on - not so much the market ups-s-downs... focus on your personal economics!
The only thing that bothers me is the prices are so high if I buy a place and the market does collapse I am in deep doo doo.
If I were you, Mulik, I'd ask myself why you think this.
If what you wish to buy is a home to live in, and you can comfortably afford the payments (plus maintenance and the transaction costs and all that), and you intend to live there for quite some time, and think that intention is reasonable given your current employment and the likelihood you could get a different job of equal earning potential without moving, if you had to, then why are you in deep doo doo if prices fall after you buy?
If the only way you can afford to buy is to stretch yourself too far, or if your situation (employment, family, whatever) means that you might very well need to move in two years, then certainly you should be cautious about buying in a deflating market.
My perspective comes from someone who has been a mortgage lender, not from someone who has ever been a real estate investor. So I tend to think of homes as homes, not as "investments," and loans as ways to buy homes. I tend to think of the "investment" part, if it happens, as gravy. Just about anyone in this country who has owned a home for ten years or more has experienced periods where that home depreciated or at least didn't appreciate. It still remained a home in that time, and as long as the mortgage payments weren't crippling, most people didn't get that worked up over it. They did, after all, still have a home.
I guess this all reminds me of locking in interest rates on a loan application, which is something I do have a lot of experience dealing with from the advice-dispensing side of things. If you consider yourself risk averse--if the thought of what you lose with a higher rate bothers you more than the thought of what you gain with a lower rate--then lock the loan. If you really think rates might go down, or you're really bothered by the possibility that you'll miss a dip, then float the loan. But don't ask your lender what to do, because your lender doesn't have the right answer for you. The lender's perspective is not yours.
Oh, man, I'm channeling dryfly!
The OFHEO looks like they're calculating their data the way I do, using apples-to-apples comparisons of current calendar month to the same calendar month the previous year. Less noisy, less volatile, fewer bad "signals."
Not as dramatic a story, of course, just a more accurate reflection of what's going on.
Sebastia
Tanta and DryFly
Thanks for your response and your opinions. I guess I have to rethink this.
The doo doo on the downside came in because I had put so much effort to save all that money over the past 5yrs. This is honest hard earned money. It felt like I might be gambling my money away if prices went down. I have lived way below my lifestyle to save up the money.
The doo doo on the upside came if I didnt buy one now and had to save some more to make the down payment.
Again I think the fault is mine being a engineer I had never tried to follow up on what effects interest rates had on assets. Being out of college recently didnt help either.
Anyway thanks for your thoughts.
Forgot to mention. here in northern virginia there is hardly any place less than 600K which is in a good shape. Even though I make good money I am still a alt A candidate.
I buy a place and the market does collapse I am in deep doo doo. If the prices go up I am in deep doo doo.
Mulik, this will likely be one of the more important joint decisions of your life. Some things to consider:
1. Is it truly a joint decision?
2. Have you both educated yourselves about the rent verses buy optionas available to you?
3. Have you both carefully considered your commitment to Northern Virginia?
4. Have you both carefully studied the housing market?
5. Can you qualify for a house and comfortably afford the payments? I foolishly listened to a real estate agent for our first home purchase and was house poor for several years (and divorced shortly after the purchase: did not listen to my wife).
6. Will you be able to get a "prime" comforming loan? If you need an interest only loan or an ARM, then maybe it is not time to buy.
7. Will you be able to rent the house for something close to your payment if you have to move and cannot sell?
8. Are you following the DC area housing blogs? One of my coworkers was considering purchasing an invesment condo in the Northern D.C. suburbs (his wife wanted it). One of the blogs talked about the exact complex they were considering. After doing some research, they are going to wait and see what happens. She is a CPA.
There is no hurry, prices are not likely to move aggressively higher in the next few months.
Again I think the fault is mine being a engineer I had never tried to follow up on what effects interest rates had on assets. Being out of college recently didnt help either.
I'm an engineer too and I learned something about this stuff - though not enough to be confused as an underwriter...
You have time.
Rob'ts points (above) are very good. Focus on Point #5. Payments over the long haul. That is the key.
That and how long will you plan to stay there. If you think you will be moving fairly soon (like within 3-5 years) then save more & do not buy. Rent & rent wisely (look for value - it is out there).
If you think you will be there a long time, then the ups-n-downs wash out. Then it doesn't really matter if you pay a little too much when you buy.
As long as you can still make the payments that is. Point #5.
Some folks on this post are asking why the OFHEO data is not showing a drop in prices. CR, didnt you once mention that the OFHEO summaries are based only on the GSE data ? Given that the GSEs dabble mostly in prime, does this explain at least part of the discrepancy ? Still learning
dryfly - how many times do you want me to debunk this "example" you keep posting? OFHEO's methodology is explained on their website universe
NJ-Bob - I believe you are on to something, and if we're right, the uptick in the ABX was exactly the wrong response to the OFHEO release. The Case Shiller index uses more or less the same methodology, but with different data. The small methodological differences might explain some it, but a) they are pretty arcane and b) I doubt it. So although I'll leave open the possibility that it's methodology, I'll bet it's data.
One data difference is timing. CS uses recorded transactions, which can lag months but usually only lag a month or so (which is why December is released in late Feb.) OFHEO uses loans bought by the GSEs, which often lag by a few months. So the first release of the OFHEO index (later revised) is weighted towards the beginning of the quarter (more Oct. loans have made it in than Dec. loans). True for CS, but to a much smaller extent. But CS shows a drop for every month of the quarter, so that can't explain more than a part of it.
Another is the fact that it's GSE data, so the national weighting is different. California especially would have less weight for OFHEO than it would for CS. But you can look at the OFHEO indices for San Diego or SF and compare them, and CS is substantially more negative. Unfortunately, you can't do a perfect job on that since OFHEO doesn't release the purchase only at the MSA level, and you have to make the wild guess that the appraisal bias doesn't differ much between markets. Still, it doesn't look like the different weighting for California explains all of it.
The third difference is that OFHEO has no jumbo loans, while CS does. If expensive houses were falling faster (in percentage terms) then CS might drop while OFHEO rises. But if you look at cheap MSAs where there aren't many jumbos (Detroit or Chicago) OFHEO and CS don't get much closer (again with the warning that the comparison isn't quite right because of the purchase-refi issue).
But the last difference (you knew there was a reason you kept reading. you did keep reading, didn't you?) is that OFHEO would have very few subprimes, while CS would have almost all of them. If prices in subprime heavy neighborhoods are falling faster than prices in other neighborhoods (makes sense if subprime originations are dropping fast, it's where the demand would be disappearing the fastest) you'd get what we see, a big divergence between OFHEO and CS. And that should scare the bejeepers out of anyone long the ABX.
"The doo doo on the upside came if I didnt buy one now and had to save some more to make the down payment."
I wouldn't sweat this.
Wait. Take your time. Watch what happens. Prices won't be going up anytime soon.
I buy a place and the market does collapse I am in deep doo doo. If the prices go up I am in deep doo doo
The doo doo on the downside came in because I had put so much effort to save all that money over the past 5yrs. This is honest hard earned money. It felt like I might be gambling my money away if prices went down. I have lived way below my lifestyle to save up the money.
I'm jack to you, so you might consider my opinion with that in mind.
You seem to have pretty much answered the question yourself. You are considering buying a house as an investment, in addition to a place to live.
I would say you think that through - because you need to invest for your future needs(retirement, kids education etc), and you need a place to live.
If your house as an investment does not pan out, can you meet your other future needs?
Just find out what you can afford to spend for housing alone, not housing & investment. If you can afford to buy on that, buy. If you cannot, rent. That "afford the payments", for me is what I can afford for shelter, with a reasonable margin for little more for owning.
Historically, house prices have just kept pace with inflation and income, long term. So prices cant go up forever disconnected to income, and prices will regress to what incomes can support. That can happen due to rising income, inflation, combination of both, whatever. So "buy now, or be priced out forever" is unfounded.
If you live in a place where the people who own, cannot afford to buy now, rest assured that prices and income will adjust, or businesses and jobs will move out.
Mort_fin -
First paragraph of the Introduction from the link you provided...
"The house price
indexes published by OFHEO -- hereafter referred to collectively or individually as the HPI -- are based on a modified version of the weighted-repeat sales (WRS) methodology proposed by
Case and Shiller (1989)."
What do you think 'weighted' & 'repeat sales' mean?
But read a little farther...
"The repeat sales method was first proposed by Bailey, Muth, and Nourse (1963), and later
extended by Case and Shiller (1987, 1989).9 This approach limits the extent to which changes in the
composition of the sample used for estimation can influence the estimated index. Utilizing information
on the values of the same physical units at two points in time controls for differences in housing
attributes across properties in the sample without directly estimating their marginal contribution to total
value. A multivariate regression is employed to account for the fact that all properties do not transact
in every period. The repeat sales method is the only approach that provides an opportunity to develop
constant quality house price indexes using GSE data. The lack of information on detailed property
characteristics in historical GSE data precludes the estimation of hedonic house price indexes."
Pretty much says it all - good link you had there.
The rest of the paper is basically how they weight and why...
But as I said SOMEHOW they track actual sales of actual homes - over time - as opposed to just summing up sales results at large per region per time bucket & comparing with a similar sample later in time.
To avoid the problem of different time scales & mix effects... they use the sampling & weighting methodology described in the paper. Not sure I understand why they do what they do but believe them that it is done.
I would say that "repeat sale" means they use prices for the same property that sell repeatedly. And I would say that "weighted" means that they give different weights to different observations. Would there be some other meanings?
halo does it again. Anonymous in the previous is, of course, mort_fi
Appreciate those details and careful (non-masking tape) posts mort-fin and dryfly.
I would say that "repeat sale" means they use prices for the same property that sell repeatedly.
But the paper doesn't say its the same properties over and over... just that they track repeat sales over time.
It does say that they correct for the various time differences... in fact they say that is part of the strength of the method in that it bridges various periods of time & corrects for sales mix. Suggest to me it isn't just the same homes repeatedly selling.
This is in part why the index damps & smooths and miss a hard change in market - either up or down.
I don't have any idea how many properties they use in their tracking samples... nor how often they use the same ones over (if they do - I assume they do if homes they are tracking or have previous data on happen to sell in the period of interest)... and I don't understand how they weight or how they then extrapolate the results of this sampling to the whole population... but I know they do & that is sufficient for me.
dryfly - why exactly would bridging various periods of time and correcting for sales mix result in damping and missing turning points?
Aren't those exactly the things that you want an index to do?
er - by 'those things' I meant bridging and correcting, not damping and missing .
Mulik,
For me it's a simple risk-reward comparison. There's just no reward to compensate for the risk of being underwater to the tune of six figures.
Aren't those exactly the things that you want an index to do?
Yes - IF they could get an instantaneous 'snapshot' without the sales mix error. But they can't. if they go with a 'snapshot' the noise hides the changes too but instead of not reporting them it 'over reports' in a series of false negatives.
So they use some kind of 'transaction tracking' algorithm to reduce the sales mix variation... but by doing that they dampen the index's response to rapid changes or inflections... in effect because of the bridging.
And if they ONLY used homes that sold rapidly & often... that too would skew the sample mix. Houses that sell often, sell often for a reason... it isn't random... they need randomness & a large tracking sample size to make the thing work.
In most cases (most periods) bridging would 'help' (or at least not hurt) because most of the time price curves of big lumbering markets like this are pretty well behaved & smooth to begin with so having multiple durations (weighted somehow) would tend to reduce noise (reduce sample error).
But it will make it much harder to pick up an abrupt REAL change in the signal one way or another because you bridge over the spike or dip. Eventually it shows up... but lagged & muted.
I don't pretend to know all about how they do it... but I gather from the piece you posted & other stuff I've read elsewhere... that their strategy is...
(1) track actual transactions over time for a sample population they have data on (the repeat sales)
(2) adjust & weight results from transaction tracking for sample mix & duration variations (the weighting)
(3) somehow project weighted sample results over entire 'universe' of housing stock to generate the index
I'd love to know more, the article doesn't explain the 'tactics'... just the 'strategy'.
And obviously I might be wrong - but that is how I read it.
mulik:
here is a link to mid altantic OFHEO price appreciation vs inflation. basically this gives a person an excellent guide when to buy RE relative to the rate of inflation.
Mid Atlantic FHFA Home Price Appreciation Tracker
dryfly - your first paragraph doesn't make any sense, your second paragraph simply repeats your assertion, your third paragraph starts with a hypothetical that they don't do, and your fourth, fifth, and sixth paragraphs just repeat another assertion.
The paper does provide the tactics as well as the strategy. They run a regression with the dependent variable as the log of the sales price, and the independent variables are a series of indicator variables that are set to -1 in the quarter of the first sale, 1 in the quarter of the last sale, and 0 for all other quarters. That's right there in the paper.
mort where do they pull the data from... the pool of sales data?
halo's prevented me from posting this 3 times now. let's try one more time.
The page cannot be displayed
OFHEO has access to this information by virtue of its role as the federal regulator responsible for ensuring the financial safety and soundness of these government-sponsored enterprises. Chartered by Congress for the purpose of creating a reliable supply of mortgage funds for homebuyers, Fannie Mae and Freddie Mac are the largest mortgage finance institutions in the United States. Their combined mortgage records form the nation's largest database of mortgage transactions.
the detailed description of the data is on p. 4 of the above link to the research paper, and the description of the regression used to generate the index is on p. 7
Right... They have a big pool, thats why they use regression.
Then the the second question is are the quarters (the -1,0,1's) all in the same quarters? If NOT in the same quarter then that's your 'bridging' right there...
In effect it turns into a very large vector 'matrix' problem.
The regression is just the way they process gazillions of transactions spread out all over time... They have a huge sample... I just used two in my 'simple example' - one long, one short.
I doubt they use the same homes over & over again in their pool... waiting for them to 'resell'. Instead they probably mine the GSE databases for transactions and then have some kind of selection criteria to randomize.
That is your Step 1 in my post above.
That still doesn't tell me how they weight for time or mix... but they do, says so.
"Then the the second question is are the quarters (the -1,0,1's) all in the same quarters? If NOT in the same quarter then that's your 'bridging' right there..."
you know, I can't even begin to parse that paragraph
Mulik,
I am watching NoVa as well, Loudoun. I think you probably focus on Fairfax. But anyway,as Tanta and Dryfly said, a home is a home, a place to live and raise kids. As to the market in NoVa, here is a blog. Blogger updates local market everyday. Good information.
Northern Virginia Real Estate Guide Blog
you know, I can't even begin to parse that paragraph
mort - they are building a matrix of transactions... from GSE databases. -1,0,1s... in fields for the quarters for which previous purchase, no purchase & current purchases (respectively) occur. Right?
Then fit via regression.
So are all the 'previous purchases dates... the occurances of those (-1)s always in the same quarter? Or are they spread out into the past for different homes & different transactions?
Example: we are in 1Q07... say we want OFHEO for 4Q06 (quarter just passed)... do we ONLY look at transaction with previous sales in one quarter (say arbitrarily, 4Q05)?
That would result in a matrix with all the -1's in 4Q05, all the 1's in 4Q06, and zeros everywhere else. Matrix would look like this:
Past qrts to Current qtr
0...0(-1)000:1
0...0(-1)000:1
0...0(-1)000:1
So on through the set of all transactions in the matrix...
Or do they use multiple transaction durations in the sample set (samples where the previous purchases could have occurred at many times in the past)... then the matrix would look something like this:
000(-1)00:1
0(-1)0000:1
0000(-1)0:1
(-1)00000:1
With the above example only looking back six quarters... could look back lots farther.
If the transaction times of the previous purchase in the sample set are spread out over time then you will have bridging and as a result smoothing.
If not - if they have a frozen duration period (and keep it real short) they will have much less bridging & less smoothing but will then have sample bias error (by limiting their sample set to only those house that flip often).
From reading the article - and watching how slowly OFHEO index moves - my guess is they bridge quite a lot.
again, the creation of the data is described on p. 4. As the paper indicates, and as I've indicated in past threads, they have multiple durations. they use all transaction pairs in data that goes back to 1975.
If by "smoothing" you mean they only produce 1 number per quarter, then how would you produce an index any other way? If by "smoothing" you mean that the amplitude of the index is consistently less than reality, then why would "bridging" as you call it produce "smoothing." You've repeatedly asserted this, but never claimed a reason.
The quarterly index has gone from around 4% per quarter to about 0.5% per quarter in about 2 years. That's slow? I'd hate to see fast. Do you honestly believe that housing was going up substantially faster than 4% per quarter as a US average 2 years ago, and that the index substantially understated it??????
let me modify that slightly. you've never claimed a reason consistent with the index being created via a regression on log prices. back when you were stating that they took some sort of average over durations you almost got to the point of claiming that averaging these averages produced the smoothing, but now that we're past that, what mechanism do you assert causes this "smoothing."
Hey Mulik, I forgot to mention that we have software jobs here in Omaha. System integrators: Weblogic (with Plumtree), Oracle, Websphere. J2EE programming is a plus. Security clearance a plus but not critical as long as you can get one eventually. Salaries are good because folks don't like to move here. Technical market is solid (insurance, Cargil, Union Pacific, DoD). Schools are great. Housing is inexpensive. I've lived and worked in CA, WA, and CO and, other than the weather, I like Omaha the best. I walk to work from a brand new house. We chickened out on building a large house on the executive lot across the street from the plant: for $400K you can buy our lot and build a McMansion. For a lot less you can buy a nice starter house. We will not likely retire to Omaha since our family is on the West coast, but it is a great place to work and reaise a family. When I was in the service, I spent two training tours at Ft. Belvior- I really like NoVa, but there are sooo many people now and it is soooo expensive.
If you (or anybody) has the skills and interest, email me at robert.monical@mslco.com.
Zip code is 68133 if you want to check out house prices. I like this site:
NPDodge Omaha Nebraska Real Estate and Council Bluffs Iowa Real Estate
My wife likes this one:
CBSHOME Omaha Real Estate Council Bluffs Real Estate
but now that we're past that, what mechanism do you assert causes this "smoothing."
The bridging causes the smoothing.
Take a look at my early example... Say there are two houses selling right now... both for $250K (just a convenient coincidence).
House #1 was previously bought 20 years ago for $50K then sold this year for $250K... Net gain.
House #2 was previously bought last year for $255K and sold this year for $250K... Net loss.
1: $250k-50k/20 = $10k/year.
2: $250-255k/1 = -$5k/year.
What does that tell us about housing prices at this very moment? Does it tell us houses have gone up or houses have gone down?
If you don't like using houses like House #1 in the dataset and only use houses like House #2 because #2's previous purchase was more recent, then you have made a selection bias right from the get go against houses with ownership stability.
If you include House #1 in the data set... then you have 'bridging problems'... because maybe House #1 went down in value last year too, but it is unknown since there was no transaction to uncover it.
If you use a lot of homes like House #1 in a large data set you will smooth out the response because of the hidden data that never gets captured. In that 20 year period prices went up and down quite a lot but house #1 missed it all.
If you throw out all homes like House #1 and only look at recent transactions you get a skewed sample for another reason BECAUSE houses like House #2 that sell often are not necessarily representative of the entire universe of all houses. We all have homes in our neighborhoods that sell often - the reason isn't always 'random'.
I believe OFHEO includes some if not a lot of homes like Home #1 - though not necessarily with twenty year old transaction histories - as well as those with the most recent transaction histories. I believe they use a staggered mix to try and reduce the sample bias error.
I believe they try and reduce the bridging errors by weighting transactions histories differently with those with more recent transactions histories higher.
That is how I interpret 'weighted repeat sales' from the article.
In short - if they reach back into the past to multiple start times they will have smoothing (signal dampening). If only because of the lost pricing information they passed over.
If they only use short selling periods (all recent) they will have sample bias & noise.
I believe OFHEO tries to split the difference between these two risks and I believe they do a good job. I don't know a better way to get a meaningful 'global number' but believe there are still issues with the index nonetheless... the biggest being the smoothing (or signal dampening).
Interesting discussion of the innards of the OFHEO index, but I still can't get past the fact that it is not really a sales index, but is rather a mixture of sales and appraisals for refinancing. In today's market, it's quite easy for me to find an appraiser to tell me my house is worth, say $300k, but quite impossible to find a real live buyer who would pay that. Unsurprisingly, the purchase-only index is lower than the combined index, which is the headline number. If you take the ratio of the HPI index annual change to the purchase only index on the spreadsheet you can download from OFHEO, the HPI index starts to diverge from purchase only in 2004 3Q. The ratio has gone from 1.03 in 20042Q to 1.43 in 20064Q, indicating that appraisals are less and less anchored to reality, and that the headline HPI is less meaningful.
szara - the MSM only talks about the purchase with appraisal refi index, but OFHEO also publishes on their website and includes in their press release the purchase only index. I tend to ignore the index discussed in the MSM and look only at the purchase only index.
dryfly - you're welcome to believe what you want about how OFHEO calculates its index, but the document on their website that I posted the link for contradicts much of what you seem to believe. But you are welcome to believe it anyway.
Since you've reposted your example, I'll repost my counterexample
;
dryfly - if they had only one observation you'd be right, there'd be no way to know the change between year 19 and year 20. But let's say they had two observations, one that goes up 220% in 20 years, and one that goes up 245% in 19 years. It is then the proverbial piece of cake to figure out that the longer observation represents a fall of about 10% in the 20th year. And they actually have more than two observations. They have millions of observations spanning various parts of the 20 year period. Parceling out that 20 year gain between the first 19 and the last one is a fairly trivial excercise in minimizing the sum of squared residuals.
mort_fin | 02.26.07 - 10:19 pm
halo again decides that I should be mort_fin in other threads, and anonymous in this one.
szara - there is a discussion of this phenomonon in the press release.
mort your 'contradiction' doesn't address any of the signal dampening issues I mention. Consider:
if they had only one observation you'd be right, there'd be no way to know the change between year 19 and year 20. But let's say they had two observations, one that goes up 220% in 20 years, and one that goes up 245% in 19 years. It is then the proverbial piece of cake to figure out that the longer observation represents a fall of about 10% in the 20th year. And they actually have more than two observations. They have millions of observations spanning various parts of the 20 year period.
Your 'contradiction' still doesn't address the smoothing due to different transaction durations for different units even if 'millions of them' are included.
Remember they track same unit repeat sales over different time periods - to avoid the issue of sales mix. So what happens to another house isn't necessarily reflective in what happens to THIS house. Else they would just use a 'global' sum & mean.
Think of each of these million transactions as a separate line each with its own 'slope'... sum all the millions and you get:
X1(t21)-X1(t11)
X2(t22)-X2(t12)
X3(t23)-X3(t13)
..........
Xm(t2m)-Xm(t1m)
With each Xi being a different house, each line a different transaction and each tij a different time j (2 being recent & 1 being previous) for each home i.
It doesn't 'look' like this because they use the -1,0,1 format but is the same math... an array of separate transactions.
From a geometric perspective, the regression is fitting all those lines to one curve... the OFHEO index. It is quite damped.
in the example that I gave the index isn't "smooth." It rises for 19 years, and tanks in 1.
You keep asserting that the index is fitting slopes, and that this causes smoothing. But you're presenting no evidence that the index is fitting slopes, and the paper describing the creation of the index explicitly states that they are fitting endpoints - all the independent variables between the dates of sale are zero. For a property that rises 21% over two years, a pattern of 10%, 10% does not fit the observation any better than does a pattern of 21%, 0%, or 0%, 21%, or for that matter, -10%, 34% or 34%, -10% (give me a few basis points for rounding). If observations on sales that occur in between this property's sales indicate that a pattern other than 10%, 10% fits better, than their is nothing in the fitting of this observation to bring the result back towards 10%, 10%.
But you're presenting no evidence that the index is fitting slopes, and the paper describing the creation of the index explicitly states that they are fitting endpoints
Yes... but the end points 'connect' a transaction history... end point 'now' minus end point 'then'. That results in a slope effect over the intervening period for THAT transaction history. Sum & fit a big number of them and you get a smoothed & dampened curve. No way to get around it using this algorithm.
Now if all they did is look at endpoint 'now' and exclude all previous 'thens'... then look at endpoint 'then' and exclude 'now', that would result in little more than two global sums & averages.
The sum (& distribution) of {Xi(t2i)-X(t1i)} can be very different from the sum (& distribution) of {Xi(t2i)} minus the sum (& distribution) of {Xi(t1i)}... The first is a vector process with slope the latter a scalar process.
I believe because they are tracking repeat sales of the same units the process is far more like the former than the latter...
Oh and the 'evidence' of this is in the regression fitting of the (-1,0,1) matrix... that is in effect the equivalent of fitting the vector array made up of all {Xi(ti2)-Xi(ti1)} with the Xi(ti2)'s corresponding to the 1's and the Xi(ti1)'s corresponding to the -1's... the 0's all the intervals before & between.
"The sum (& distribution) of {Xi(t2i)-X(t1i)} can be very different from the sum (& distribution) of {Xi(t2i)} minus the sum (& distribution) of {Xi(t1i)}... "
It is algebraically impossible for the sum of {Xi(t2i)-X(t1i)} to be different from the sum of{Xi(t2i)} minus the sum of {Xi(t1i), as you assert.
And they are not summing endpoints. They are minimizing the sum of squared residuals between actual endpoints and endpoints predicted by a set of index values. That's what a regression is.
Say you have 3 transactions. One goes up 20% between year 1 and year 2. The second goes down 24% between year 2 and year 3. The third changes by 0% between year 1 and year 3. Explain to me a) why the index wouldn't register a 20% increase for year 1 and a 24% decrease for year 2 or b) why those index values would represent smoothing.
It is algebraically impossible for the sum of {Xi(t2i)-X(t1i)} to be different from the sum of{Xi(t2i)} minus the sum of {Xi(t1i), as you assert.
I have to think about that one - it wasn't a good example. I know you are right if X is linear & there are no sample selection issues for X(t) period to period.
But if X(t) is nonlinear, or if there are sample selection issues creating different samples period to period, then I'm not so sure.
The price/valuation curve OFHEO is trying to map probably isn't linear.
And there are sample selection issues (the units that sell vs. those that don't sell spread out over different intervals).
Regardless it was still a bad point on my part.
I know you know a lot about regression & statistics in general... can see that. But I think you are missing or underestimating the impact from the hidden appreciation of the previously unsold units carried forward that don't show up until that particular unit is sold.
Until a transaction takes place that appreciation is 'zeroed' out in the regression.
Realize I am not interested in the OFHEO index vs some other index but rather OFHEO vs what is really happening in the actual real estate market & does OFHEO always measure that well. That is what you want an index to do - track reality.
I think it lags & is dampened compared to what is happening in the 'real world' and I think we are seeing that now with OFHEO still increasing but real markets probably already heading down.
Even though I don't think it is a terrible index, I think OFHEO has missed this turning point. I believe it will also miss the initial change toward a future rebound - whenever that happens.
But I clearly don't have a complete enough grasp of the mechanics to explain why I 'feel' this way. I have to read more & crunch some numbers before I will be able to do that.
Have you any other references besides this paper - ones where they actually walk through the calculation with data sets?
http://www.ofheo.gov/media/archive/docs/working/02-2dreimancross.pdf
http://www.ofheo.gov/media/pdf/workingpaper033.pdf
http://www.ofheo.gov/media/pdf/061appraisal_bias.pdf
one or more of those three might help. Also, if you have access to academic searches, an article by Brad Case (not related to Karl Case) and Henry Pollakowski in the mid 1990's in, I think, the American REal Estate and Urben Econ Jrnl (but I might have that wrong) walked through a lot of these issues.
And properties aren't "zeroed" until they are sold. They are not included in the regression at all until they sell.
I can't help but guffawing at this thread.
Dryfly and Mort_fin have succeeded in creating the most techncally involved discussion on the most arcane of differences, in the Blogosphere.