Lender ROI on Prosper Select loans declines for sixth consecutive month

MNH Report #6


Since August 2007, Prosper management has tried to deflect criticism of high default rates and low return-on-investment by calculating what it calls the Prosper Select Index, and publishing that index in a monthly People-to-People Lending Market Survey. Rather than considering the ROI on all Prosper loans, the Prosper Select Index encourages investors to focus on a subset of the market which, according to Prosper, should produce a more acceptable profit for lenders.

Prosper defines the Prosper Select Index as

the estimated average annual return on invested principal, based on actual delinquency performance to date. The Prosper Select Index includes AA - E credit grade loans for borrowers whose credit reports at the time of application indicated zero current delinquencies, three or fewer credit inquiries, and a debt-to-income ratio of 40 percent or less. The annual return period reflects loans originated in the twelve month period ending one month prior to the observation date.

The Prosper Select Index for each month is also broken down into three smaller subsets: Prime Select (A and AA loans), Near Prime Select (B through D), and Sub-Prime Select (E). HR loans are not included on the Select Indexes.

For each month’s Prosper Select Index, the observation date is the last day of the month. For instance, the August 2007 Prosper Select Index, published in mid-September, calculated the ROI on loans meeting the selection criteria which originated from August 1, 2006, through July 31, 2007, with performance observed as of August 31, 2007. Less than one-third of the Prosper loans originated during that 12-month period (32.1% by dollar amount) are included in the calculation of the Prosper Select Index; Prosper doesn’t publicize the ROI on the remaining 67.9% of loans funded during that period.

On February 12, Prosper released its People-to-People Lending Marketplace Survey for January 2008, revealing that lender ROI on the loans included in the Prosper Select Index has declined in each of the six months since the Index first appeared.

For August 2007, the Prosper Select Index was 10.31%. By January 2008, it had dropped more than two percentage points, to 8.19%. Here are all of the published monthly Prosper Select Indices:

August 2007 – 10.31%
September 2007 – 9.75%
October 2007 – 9.28%
November 2007 – 9.13%
December 2007 – 8.34%
January 2008 – 8.19%

The decline occurred across all credit grades. The Prime Select Index declined from 9.41% to 8.75%, Near Prime Select Index slipped from 10.73% to 7.90%, and Sub-Prime Select Index plummeted from 14.95% to 4.93%.

(Interestingly, the first four Lender Market Surveys are included in the "Press Releases" section of the Prosper.com site, while the December and January versions have been relegated to its blog.)

What caused this substantial decline in lender ROI for Select Index loans? It’s hard to say, but the cause appears to be an increase in projected losses due to default rather than a drop in borrower interest rates. During the same period, borrower interest rates fell only 0.24% (10.15% to 9.91%) for prime borrowers, 0.03% (16.83% to 16.80%) for near prime, and actually increased 2.51 percentage points (25.88% to 28.39%) for sub-prime.

My best guess – and it is only a guess – is that Prosper’s increased emphasis on standing orders, portfolio plans, and numbers-based lending has led lenders to blindly fund loans whose flaws which do not appear in the credit summary data, but which lenders might have avoided had they considered all of the available information. Another possibility is that the evolution of Prosper’s algorithm for calculating projected defaults has led to more pessimistic projections for identical loans.

There's more to this story than the decline in lender ROI over the past six months; the methodology by which Prosper calculates the Prosper Select Index leads to misleadingly optimistic results, for at least three reasons.

First, as noted above, it excludes a majority of Prosper loans from its calculations, and, as such, does not reflect the ROI experienced by a majority of Prosper lenders. Second, it appears that the estimated losses due to default are a moving target, changing retroactively as Prosper acquires additional data and refines its algorithms. Third, because the observation date is only one month after the end of the loan origination period, many of the loans included in the calculation have not yet had an opportunity to go late or default, leading to overly optimistic performance estimates.

To demonstrate the effects of the second factor, I attempted to replicate Prosper’s calculation of the August 2007 Prosper Select Index using Prosper’s own performance page on February 17, and again on February 19.

Setting loan origination dates to August 1, 2006 to July 31, 2007, DTI to 0.00% to 40.00%, current delinquencies to zero, credit inquiries to zero to three, and observation date to August 31, 2007, the performance page returned the following ROI by credit grade on February 17:

AA – 8.61%
A – 9.61%
B – 8.53%
C – 8.41%
D – 13.15%
E – 14.58%

Weighting the data by dollar amount of loans for each credit grade, I arrived at an August 2007 Prosper Select Index of 9.61%, significantly below Prosper’s published figure of 10.31%. (Weighting by number of loans yielded a Prosper Select Index of 9.86%.)

There are several possible explanations here, the most charitable of which is that Prosper has changed its algorithms for calculation of default rate since August 31, 2007, and applied those changes retroactively to the data. As a result, even though the observation date stayed the same, the change in calculation date produced different results. (I confirmed this theory by running the test again, two days later. The results – for net defaults, adjustment, and average annual return - were slightly different.)

(I will defer to other, more statistically astute, observers regarding the appropriateness of the use of the term “observation date” when the data observed as of that date is subject to retroactive change. It gives me a headache just thinking about it.)

It appears that Prosper’s estimates of net defaults have become more pessimistic over the past six months, although this effect is not sufficient to account for the overall decline in the Prosper Select Index since August.

I examined the third factor by changing the observation date to February 15, 2008, but leaving all other criteria intact. (Calculation date was February 17.) The performance page returned the following ROI estimates for loans which would have been included in the August 2007 Prosper Select Index:

AA – 8.54%
A – 6.14%
B – 5.64%
C – 5.99%
D – 9.05%
E – 7.18%

Extending the observation date by 5.5 months resulted in a significant decrease in estimated ROI for each credit grade, with the most pronounced reduction in the lower credit grades.

Using the extended observation date, and weighting the data by dollar amount of loans for each credit grade, I arrived at a August 2007 Prosper Select Index of 7.03%, more than three percentage points below Prosper’s published figure. (Weighting by number of loans yielded a slightly higher figure, 7.14%.)

Again, the February 15 observation date was only 6.5 months after the end of the 12-month loan origination period, and only 5.5 months later than the observation date used by Prosper to create its published ROI of 10.13%. If the estimated ROI for Prosper Select loans dropped three percentage points during those 5.5 months, one can only wonder what the ROI would be at the end of the three-year term of the notes.

The same three factors which resulted in the overly optimistic August 2007 Prosper Select Index which Prosper published in its first People-to-People Lending Market Survey also affect the estimated ROI data appearing elsewhere on Prosper.

For instance, Prosper’s home page advertises 9.49% – 12.81% returns. But the footnote shows that this ROI is based on (1) specific credit criteria, (2) obsolete default prediction algorithms yielding data which cannot be replicated, and (3) an observation date one month after the end of the loan origination period:

Rates of return shown are the average annual returns on Prosper loans originated between 7/1/06 and 10/30/07 to borrowers with AA - C credit grades who requested automatic funding, have 0 - 1 current delinquencies and 0 inquiries in the last 6 months on their credit record, as of 11/30/07. For more information click here.

Clicking on the link takes one to the Prosper performance page, where the 9.49% - 12.81% ROI figures cannot be replicated. It is worthy of note that 2.5 months have passed since the observation date used to generate those numbers, so Prosper could have updated these advertised returns with a later observation date and its current default projections, but chooses not to.

Prosper’s Portfolio Plans, on which many lenders rely to make lending decisions, are accompanied by a similar footnote:

Estimated average annualized loss rate based on the historical performance of Prosper loans for borrowers with similar characteristics, originated between Jun-01-2006 and Oct-31-2007, measured as of Nov-30-2007.

Give that a 5.5 month change in observation date and a 6.5 month change in calculation date produces a three percentage point drop in estimated lender ROI for Select Index loans originated between August 1, 2006 and July 31, 2007, lenders should carefully consider whether the ROI figures on the home page and in the portfolio plans are similarly flawed, and should also be adjusted downward.