Analytics blog from Avalon Business Systems. To find out more about our analytics offerings and SPSS Modeler please CLICK HERE.

    A Press Release on Opioids: "Record Number of New Jersey Doctors Disciplined in 2016 as State Ramps Up Efforts to Curb Opioid Addiction"

    Eugen G Tarnow  March 21 2017 10:28:23 AM
    By Eugen Tarnow, Ph.D.
    Avalon Business Systems, Inc.
    http://AvalonAnalytics.com

    The New Jersey Department of the Attorney General issued a press release earlier this month about 31 medical doctors disciplined for opioid issues (you have to ask for the actual 31 names, they are not in the press release but you can find them on the bottom of this blog). It states "Purging the medical community of over-prescribers is as important to our cause as busting heroin rings and locking up drug kingpins.”

    Well, if that is the case, few kingpins would be locked up and then for only a short time.

    All but 5 of the doctors had prescribed opioids within Medicare.  The average number of Medicare prescriptions for these doctors were 650 per year.  327 NJ doctors had prescribed more than that within Medicare which suggests a conviction rate of 10%.

    The majority of opioid sanctions were temporary: 61%.

    The doctors sanctioned for opioids have somewhat more sanctions than the average sanctioned doctor (1.9 versus 1.5).  One doctor had 8 sanctions altogether - 7 previous sanctions.

    That the state is indeed ramping up its efforts is shown by a graph of the number of doctors sanctioned by month covered by the press release - it is increasing:
    Image:A Press Release on Opioids: "Record Number of New Jersey Doctors Disciplined in 2016 as State Ramps Up Efforts to Curb Opioid Addiction"

    However, the total number of sanctions by the medical board is the same over the last ten years, about 114 (which probably works out to about 0.4% of all active physicians in NJ):
    Image:A Press Release on Opioids: "Record Number of New Jersey Doctors Disciplined in 2016 as State Ramps Up Efforts to Curb Opioid Addiction"

    There were also personal tragedies among the doctors - at least one had a child who overdosed on the doctor's opioid pills.

    Here is an interesting question: will anybody sue these 31 doctors and be able to collect from their malpractice insurance?  Will the insurance companies then represent these doctors?

    Many of the doctors had multiple versions of their names, one even had the first and last names switched in the different data sets.  Furthermore, not all sanctions could be found on the website of the State Board of the Medical Examiners (http://www.njconsumeraffairs.gov/bme/Pages/actions.aspx). One wonders whether data quality issues make it harder to catch perpetrators.

    Final 2016 CDS Case List.docx
    Comments Disabled

      The Art of the Deal: Sweden pays the least for drugs in all of Europe (and much much less than the US)

      Eugen G Tarnow  February 24 2017 03:06:02 PM
      By Eugen Tarnow, Ph.D.
      Avalon Business Systems, Inc.
      http://AvalonAnalytics.com

      When looking at Sweden, there are many gems to be found.  One is how they negotiate their drug prices.  It is something that we all should take a look at.

      An Epipen is $40 and nobody forces you to buy 2 of them.  In the US a pair is $700 and you have to buy both.

      See: http://www.tlv.se/Upload/Ovrigt/Rapport_internationell_prisjamforelse_lakemedel_151215.pdf


      Image:The Art of the Deal: Sweden pays the least for drugs in all of Europe (and much much less than the US)

      For prices of particular drugs, see: http://www.tlv.se/beslut/sok/lakemedel/

      Call your politicians and your journalist friends and ask them to cover this.


      Comments Disabled

        Algebra 1 Scores & Social Group: Differences Increase from 2015 to 2016

        Eugen G Tarnow  February 21 2017 01:43:24 AM
        By Eugen Tarnow, Ph.D.
        Avalon Business Systems, Inc.
        http://AvalonAnalytics.com

        As income inequality increases, one would expect standardized test scores to do the same.  In the NJ PARCC Alegra 1 test, this is indeed the case.  Here are the 2015 scores by social group:

        Image:Algebra 1 Scores & Social Group: Differences Increase from 2015 to 2016

        And here are the differences between 2016 and 2015

        Image:Algebra 1 Scores & Social Group: Differences Increase from 2015 to 2016

        As the social group increases from A to J, the scores go up.  This corresponds to 73% of the variation:

        Image:Algebra 1 Scores & Social Group: Differences Increase from 2015 to 2016

        As always, I reserve the right to be wrong.
        Comments Disabled

          H1B Salary Distributions

          Eugen Tarnow  February 2 2017 11:45:46 AM
          By Eugen Tarnow, Ph.D.
          Avalon Business Systems, Inc.
          http://AvalonAnalytics.com

          The salary distributions of H1B workers - for single applications looks like this.  The most common salary is $60-65,000.
          Image:H1B Salary Distributions
          For applications with two workers needed at the same time the distribution narrows and, again, the most common salary is $60-65,000 per year.
          Image:H1B Salary Distributions
          In Silicon Valley the distribution for single applications is centered around $100,000:

          Image:H1B Salary Distributions
          For applications with two workers it is much lower with the most common salary between $75-80,000:
          Image:H1B Salary Distributions

          In New York City, the single application distribution is as follows:

          Image:H1B Salary Distributions

          and the two-worker application distribution is:

          Image:H1B Salary Distributions

          The most common salary in the second case is $70-75,000.  

          Cognizant typically applies for 30 workers at a time.  For those applications the salary distributions look like:

          Image:H1B Salary Distributions
          The most common salary is $60-65,000.

          The data was taken from https://www.foreignlaborcert.doleta.gov/performancedata.cfm using fiscal year 2015 and when the employer gives a range in pay, the lowest number was chosen.
          Comments Disabled

            Which Occupations Need The Cheapest H1B Salaried Workers?

            Eugen G Tarnow  January 31 2017 01:56:59 PM
            By Eugen Tarnow, Ph.D.
            Avalon Business Systems, Inc.
            http://AvalonAnalytics.com

            One might be curious which occupations the US just don't have enough workers in but at the same time are so easy to find abroad that they can be paid the minimum allowable H1B salary? The majority is programmers and analysts - no need to focus on the physical therapists (???).

            Image:Which Occupations Need The Cheapest H1B Salaried Workers?

            The data was taken from https://www.foreignlaborcert.doleta.gov/performancedata.cfm using fiscal year 2015 and when the employer gives a range in pay, the lowest number was chosen.
            Comments Disabled

              Which US Counties Have No Smart STEM People & Need The Most H1B Workers At Minimum Allowable Salary?

              Eugen Tarnow  January 31 2017 01:44:13 PM
              By Eugen Tarnow, Ph.D.
              Avalon Business Systems, Inc.
              http://AvalonAnalytics.com

              Which counties just don't have enough smart workers so they have to hire minimum salary H1B workers?  Here is a list and it includes places with the lowest STEM ability like Silicon Valley, New York, King (where Microsoft is), Dallas, San Francisco, Bergen (where I live).  

              Image:Which US Counties Have No Smart STEM People & Need The Most H1B Workers At Minimum Allowable Salary?

              The data was taken from https://www.foreignlaborcert.doleta.gov/performancedata.cfm using fiscal year 2015 and when the employer gives a range in pay, the lowest number was chosen.
              Comments Disabled

                Which Attorneys File The Most Applications Paying H1B Workers the Legal Minimum?

                Eugen G Tarnow  January 31 2017 01:30:16 PM
                By Eugen Tarnow, Ph.D.
                Avalon Business Systems, Inc.
                http://AvalonAnalytics.com

                One might be curious who are the lawyers that file the most H1B applications with the minimum legal limit?  See below.

                The data was taken from https://www.foreignlaborcert.doleta.gov/performancedata.cfm using fiscal year 2015 and when the employer gives a range in pay, the lowest number was chosen.

                Image:Which Attorneys File The Most Applications Paying H1B Workers the Legal Minimum?



                Comments Disabled

                  Which Companies Pay Less Than 1% Over Legal Minimum for H1B Workers?

                  Eugen Tarnow  January 31 2017 01:11:46 PM
                  By Eugen Tarnow, Ph.D.
                  Avalon Business Systems, Inc.
                  http://AvalonAnalytics.com

                  Which companies pay their H1B workers the very minimum?  Here is a list.  The largest number of minimum wage applications come from Indian companies.  But we have American ones as well:  PWC, Deloitte, Accenture, CapGemini, Apple, Oracle, Cisco.

                  Image:Which Companies Pay Less Than 1% Over Legal Minimum for H1B Workers?

                  Do you think our colleges are any different?  Here is a list of the colleges with the most workers with minimum pay:

                  Image:Which Companies Pay Less Than 1% Over Legal Minimum for H1B Workers?

                  The data was taken from https://www.foreignlaborcert.doleta.gov/performancedata.cfm using fiscal year 2015 and when the employer gives a range in pay, the lowest number was chosen.
                  Comments Disabled

                    Statistics of the H1B Program: Most H1B Workers Get Minimum Pay

                    Eugen G Tarnow  January 31 2017 12:18:36 PM
                    By Eugen Tarnow, Ph.D.
                    Avalon Business Systems, Inc.
                    http://AvalonAnalytics.com

                    The H1B program is a goldmine for the corporations participating.  While it is supposed to supplant the lack of STEM talent in the US, the evidence is not there.  

                    Here is the promised salaries for single and multiple H1B applications.  Note that 44% (72%) of single (multiple) applications pay less than 1% over the prevailing wage, which is the legal minimum pay.

                    The data was taken from https://www.foreignlaborcert.doleta.gov/performancedata.cfm using fiscal year 2015.

                    Image:Statistics of the H1B Program: Most H1B Workers Get Minimum Pay


                    Comments Disabled

                      School District Income Seems to Mean Little to the Poor

                      Eugen G Tarnow  January 25 2017 06:50:15 PM
                      By Eugen Tarnow, Ph.D.
                      Avalon Business Systems, Inc.
                      http://AvalonAnalytics.com

                      I had a conversation with a school board member in a New Jersey district in which the mean family income is large but economically disadvantaged students do not do well.  I decided to see if this was a larger phenomenon.

                      Indeed it was.  For example, below is shown the grade 6 PARCC math scores as a function of district family income for all children (top) and for economically disadvantaged children (bottom).  There is a strong correlation between the math scores and the district family income in general (R square is 44%) but not for the poor (R square is only 2%).  

                      It is also interesting just how expensive it is to increase the average grade by 1 point: $5,000 in mean income!

                      Image:School District Income Seems to Mean Little to the Poor
                      Comments Disabled

                        Student Loans Go Missing

                        Eugen G Tarnow  January 19 2017 12:59:59 PM
                        By Eugen Tarnow, Ph.D.
                        Avalon Business Systems, Inc.
                        http://AvalonAnalytics.com

                        Student loans are in the news because the Department of Education had a programming glitch that understated the number of delinquent student loans.

                        It is interesting to see how the student loans become delinquent.  I used the data from https://studentaid.ed.gov/sa/about/data-center/student/portfolio .

                        Student loans that are 31-90 days delinquent have a 56% chance of becoming delinquent 91-180 days, then a 57% chance of becoming delinquent 181-270 days and then a 70% chance of becoming delinquent 271-360 days.  

                        After that the loans are to be transferred into collection.  The number of loans in collection is not reported, only the number of loans transferred into collection.

                        This is where it becomes interesting.  One would think that the loans would go into collection as soon as they reach 361 days.  But after a quarter, there is no correlation between the delinquencies and the collections (all in billions):

                        Image:Student Loans Go Missing

                        Rather, if we plot the linear coefficient between delinquencies and collections as a function of the time delay, we find that it takes 6 quarters before there is a relationship between the two and then the relationship is a negative one, the more delinquencies, the fewer of them go to collection:

                        Image:Student Loans Go Missing

                        If we plot the missing loans as a function of quarter we find that it keeps rising:

                        Image:Student Loans Go Missing

                        If we add up the total missing loans after so many quarters we get:

                        Image:Student Loans Go Missing

                        Forecasting by extrapolation we get:

                        Image:Student Loans Go Missing

                        It will reach $1 Trillion in 68 quarters from now, in 17 years.  

                        As always, I reserve the right to be wrong.
                        Comments Disabled

                          NJ PARCC Algebra 1 Scores Changes Over A Year

                          Eugen G Tarnow  January 4 2017 06:49:41 PM
                          By Eugen Tarnow, Ph.D.
                          Avalon Business Systems, Inc.
                          http://AvalonAnalytics.com

                          The 2016 PARCC scores are out and they are quite interesting.  The zero score is 650 and the highest possible is 850, a 200 point spread. Below I show you the 2016 scores versus the 2015 scores in Algebra 1 for all schools.  There is a spread - the scores of those two years are not quite the same.

                          Image:NJ PARCC Algebra 1 Scores Changes Over A Year

                          If I plot the difference between the two years as a function of the 2015 score we end up with the plot below.  Overall the average scores have improved by 5, the standard deviation is 9, and the improvement is highest for the lowest performing schools.  Some schools improved by as much as 50 points but some schools declined by 40 points.  

                          Image:NJ PARCC Algebra 1 Scores Changes Over A Year

                          Some of these changes are statistical noise as can be seen in the corresponding plots of schools with at least 100 valid scores.  Again the improvement is on average 5 points but the standard deviation is a smaller 7.6.  The improvement is highest for the lowest performing schools.  Some schools improved by 35 points and some declined by 20 points.

                          Image:NJ PARCC Algebra 1 Scores Changes Over A Year

                          What could account for the large differences?  The students or the teachers?  Let's increase the number of valid scores to 200 to lower the noise from the student population further.  The corresponding plots are below. The improvement is on average 5 points and the standard deviation is 7.3 and the improvement is highest for the lowest performing schools.  Again some schools improved by as much as 20-30 points and some declined by 10-20 points.

                          Image:NJ PARCC Algebra 1 Scores Changes Over A Year

                          Since we varied the number of students and got similar results, it would seem that the difference in test scores are due to the teachers. In other words, variability between Algebra 1 teachers within the same school account for about 7 PARCC points (but can vary between -20 to 35 according to the plot just above).

                          Finally, the extra 5 points are somewhat dependent on socioeconomic characteristics, see the plot below.  "A", "B" etc. refer to "District Factor Groups" and the order is low to high socioeconomic characteristics.

                          Image:NJ PARCC Algebra 1 Scores Changes Over A Year

                          The score differences as a function of socioeconomic characteristics are shown below:
                          Image:NJ PARCC Algebra 1 Scores Changes Over A Year

                          There is no strong relationship of large declines or improvements with socioeconomic characteristics, see below:

                          Image:NJ PARCC Algebra 1 Scores Changes Over A Year

                          As always, I reserve the right to be wrong.
                          Comments Disabled

                            NJ PARCC English 11 Scores Changes Over A Year: Differences Due to Socioeconomic Factors Minimized by Excluding Elite Students

                            Eugen G Tarnow  January 4 2017 06:49:41 PM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            On September 9th, 2015 Commissioner Hespe allowed 11th grade students who were enrolled in AP or IB courses AND took the assessment associated with those courses a “waiver” from the PARCC grade 11 ELA assessment.

                            The consequences of excluding the elite students from the PARCC test were large and strongly correlated with socioeconomic characteristics, see the plot below.  "A", "B" etc. refer to "District Factor Groups" and the order is low to high socioeconomic characteristics.  

                            Image:NJ PARCC English 11 Scores Changes Over A Year: Differences Due to Socioeconomic Factors Minimized by Excluding Elite Students

                            The waiver decreased the scores of high performing schools and increased the scores of low performing schools.  The score differences as a function of socioeconomic characteristics are shown below:

                            Image:NJ PARCC English 11 Scores Changes Over A Year: Differences Due to Socioeconomic Factors Minimized by Excluding Elite Students

                            The consequence can be seen the clearest in the graph below that shows the average scores by socioeconomic group for the two years plotted against each other.  The score differences in 2016 are only 44% of the differences in 2015.  It looks as if socioeconomic factors are only 44% of what they were before.

                            Image:NJ PARCC English 11 Scores Changes Over A Year: Differences Due to Socioeconomic Factors Minimized by Excluding Elite Students

                            Another way to put it: the F test for socioeconomic characteristics yielded 29 in 2015 but only 14 in 2016.

                            As always, I reserve the right to be wrong.
                            Comments Disabled

                            Providers in Republican States Prescribe Higher Levels of Opiates Than in Democratic States

                            Eugen G Tarnow  December 6 2016 09:13:23 PM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com


                            AP wrote a story ( click here ) about how the makers of opioids lobby state legislatures very successfully.

                            I found independently that Medicare opioid prescriptions are very strongly correlated with state poverty - the poorer the state, the more Medicare opioid prescriptions.  This presumably means that the state legislatures in poor states are easier to pay off than state legislatures in rich states.  The payoff results in opioid favorable state laws; a lack of laws limiting opioids.

                            I then decided to look at whether there is a difference between parties: do states with Republican legislatures write less opioids than states with Democratic legislatures or vice versa?  

                            It turns out that states with Republican legislatures have providers that write more Medicare opioid prescriptions. This is shown in the graph below.  In each of the graphs is displayed the percent Medicare opioid claims out of all Medicare claims as a function of median household income.  The red dots correspond to Republican governors and the blue dots to Democratic governors.  The lines are the least square fits to the dots.  

                            In the top graph I find that states with Republican governors have more Medicare opioid claims than states with Democratic governors. In the middle graph we see that is even more so for Republican upper houses and, in the bottom graph, for Republican lower houses (I used the Wikipedia definition of lower and upper houses).

                            Image:Providers in Republican States Prescribe Higher Levels of Opiates Than in Democratic States

                            This result is not obvious.  Below are the same graphs for the murder rate.  It is a lot less sensitive to the median household income and the variation with party is not as consistent.

                            Image:Providers in Republican States Prescribe Higher Levels of Opiates Than in Democratic States

                            A one-way ANOVA showed that there was a significant difference between Democrat and Republican Lower Houses for opioids (F=16.7, p<0.001) but not for the murder rate (p=0.8), Upper Houses for opioids (F=4.6, p=0.015) but not for the murder rate (p=0.67), Governor for opioids (F=4.23, p=0.045) but not for the murder rate (p=0.58).
                            Comments Disabled

                            Do Math Supervisors Influence PARCC Scores?

                            Eugen G Tarnow  November 19 2016 09:05:16 AM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            I have previously written about how difficult it is for a district to get good math teachers (as reflected in the weak correlation between district parental income and PARCC scores).  

                            This time I asked the question: do math supervisors matter for the PARCC scores?

                            If they do, then if the math supervisor is good, all the math scores would tend to be good and if the math supervisor is bad, all the math scores would tend to be bad.  In other words, there should be a strong correlation between the math scores of a school district if the math supervisor matters.  If the math supervisor does not matter, the math scores would be correlated with each other no more than with some other subject.

                            I compared the correlations within the PARCC math scores with the correlations between the PARCC math scores and the NJBCT biology score.  I only included schools with complete scores, see http://njschoolperformance.com.

                            The result was: no statistical difference between the two.  In other words, biology is as correlated with the math scores as math scores are correlated with each other.

                            Thus it seems that math supervisors do not influence the PARCC scores.
                            Comments Disabled

                            Globalization helps unfree nations become free but free nations move the other way

                            Eugen G Tarnow  September 21 2016 09:05:52 AM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            Here are two charts I find interesting in the midst of the discussion of globalization.  I used the Index of Economic Freedom (Heritage Foundation and the Wall Street Journal) to see what globalization does to free nations.  The left chart shows that GDP per capita tends to correlate strongly with freedom.  BUT the right chart shows that during the last three years, freedom did not correlate with the GDP ratio.  So at least in one measure globalization is not favoring free countries.

                            Image:Globalization helps unfree nations become free but free nations move the other way

                            The average freedom score, weighted by population, has increased by 0.7 points in the same time period, a positive sign.  However, freedom is moving from free to unfree countries: the difference in freedom correlates negatively with the freedom score.

                            Image:Globalization helps unfree nations become free but free nations move the other way

                            Something to discuss at this week's meeting of the United Nations General Assembly, no?

                            And here is how we in the United States have lost our freedom since 2009:

                            Image:Globalization helps unfree nations become free but free nations move the other way
                             
                            Comments Disabled

                            The eight best school districts of NJ - a district for every budget

                            Eugen G Tarnow  September 13 2016 02:09:05 PM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            When a family is looking to settle down in NJ they are often trying to figure out what is the best school district they can afford.  

                            We ranked all the school districts based on their PARCC test data (http://NJSchoolPerformance.com ), including only school districts with complete data.  

                            For every income level there is a highest ranking school district and here is our list:

                            School district Avalon ranking Median family salary
                            PHILLIPSBURG TOWN (Warren County) 62 $52,484
                            JERSEY CITY (Hudson County) 54 $64,419
                            FORT LEE (Bergen County) 46 $87,289
                            NEW MILFORD (Bergen County) 17 $97,639
                            LEONIA (Bergen County) 14 $100,283
                            EDISON (Middlesex County) 12 $105,670
                            MAHWAH (Bergen County) 4 $120,107
                            BERNARDS (Somerset County) 1 $165,714



                            Comments Disabled

                            Educational Quality Control Lacking - New Jersey School District Subject Knowledge Test Results & Family Income

                            Eugen G Tarnow  September 6 2016 09:58:15 AM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            We have seen that family income determines 45% of the Avalon school ranking of New Jersey school districts.  

                            But income does not determine specific subject knowledge.  In other words, you do not get the bang for the buck you might expect in each subject.  

                            In Figure 1 I show you how much of the subject knowledge is determined by the family income in NJ school districts.  Math knowledge increases from Algebra 1 to Algebra 2 perhaps because the Algebra 1 score is influenced by both middle school and high school students which might make the teachers difficult to supervise.  The trend is the opposite in English, presumably because parents are strongly involved in English 9 and then give up their involvement resulting in an almost random English score in grade 11.

                            In other words, while district family income determines the AVERAGE score over all subjects, the SPECIFIC scores are left to chance.

                            If you think school is important, it is time to get involved.  A talk with a low performing department supervisor might be a good beginning.

                            Image:Educational Quality Control Lacking - New Jersey School District Subject Knowledge Test Results & Family Income
                            Figure 1.  Importance of district income in determining subject knowledge.

                            Comments Disabled

                            New Jersey School Districts Rank Determined By Family Income But Bargains Still Remain to Be Had

                            Eugen G Tarnow  September 5 2016 01:05:49 PM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            I performed probably the most comprehensive ranking of New Jersey school districts (see http://NJSchoolPerformance.com ).  Since SAT scores are determined by family income and less by the school performance, we ranked the New Jersey school districts based on the standardized test PARCC high school scores and the corresponding scores from middle school honors programs.

                            I expected there to still be a correlation with family income but I was taken back when I found that family income made up 45% of the ranking (see figure below).

                            Nevertheless, the remaining spread is large enough for there to be bargains to be had.  If you want to live in one of the top ten ranked school districts, expect to have to make about $103,000-$211,000 per year. (NOTE: the family income does not necessarily determine the amount of money the school get to spend per student - that's a different topic.)

                            And if you live in school district that has low ranking despite being in a town with high family incomes - talk to the corresponding board of education (and good luck with that!).

                            Image:New Jersey School Districts Rank Determined By Family Income But Bargains Still Remain to Be Had
                            Figure. Avalon ranking of school district versus median family income.

                            Comments Disabled

                            NJ School Districts: SAT Scores, PARCC Scores, Suspension Rates & Family Income

                            Eugen G Tarnow  August 26 2016 10:44:50 PM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            Many families live in particular New Jersey towns for the school districts and at least some select the school district based on the SAT scores. However, a higher SAT school district carries with it financial hardship.  One can make a more optimal financial decision if one could tease out what part of the SAT scores are based on school performance and what is based on individual family spending on SAT coaching – since the coaching can take place in any school district.
                            For this purpose, I am able to exploit a new standardized test in NJ public schools, the PARCC test, which is relatively opaque to parents.  It was designed with a lowest possible score of 650, presumably to avoid parental shock, is accompanied by frequent assurances to students that the PARCC test scores are “confidential”, the student reports are almost unreadable (including a demographically matched set of school districts that do not match the local area) and the schools do not discuss their performance reports.  This suggests that the PARCC results reflect the combination of the quality of the school education and the quality of the students and less so parent involvement.  The SAT score, on the other hand, alerts parents and worries students as it carries great importance in college admission, suggesting that the SAT score reflects, in addition to the quality of the school education and the quality of the students, more student effort and parental input.

                            One important part of the student effort and parental involvement to increase the SAT score is paid coaching.  It would seem reasonable that the access to paid coaching is income based.  The lowest incomes should correspond to little paid coaching while the highest incomes should correspond to the best paid coaching available. Thus if one removes the effects of the school environment via the PARCC scores, the remainder should reflect the effect of best paid coaching available (this reasoning is similar to that of Zwick & Green, 2007).

                            Fig. 1 displays the effect of family income on PARCC and SAT scores as well as the student teacher ratio and the student suspension rate.  The student teacher ratio has a very small effect and will be neglected.  Income differences affect some subject scores more than others.  Note the trend for English – as high school progresses the correlation with income decreases.  The correlation with income is also low for Algebra 1. The strongest PARCC test correlation with income is the overall average.  The student suspension rate is also strongly correlated with income.
                            Image:NJ School Districts: SAT Scores, PARCC Scores, Suspension Rates & Family Income
                            Fig. 1 PARCC and SAT test score correlations with median family income.  


                            In Fig. 2 is displayed the correlation with income of the SAT score with the PARCC average and the student suspension rates removed separately and together via a linear regression.  In Table 1. is displayed the SAT score difference for incomes between $50,000 and $170,000. The unaltered difference is 187 and the difference with the effects of the PARCC average score and student suspension rates is 74.

                            Image:NJ School Districts: SAT Scores, PARCC Scores, Suspension Rates & Family Income
                            Figure 2.  Correlation of SAT score with family income removing effects from suspensions and PARCC averages.


                            Removing suspensions Removing PARCC Average Removing both PARCC Average and suspensions
                            SAT score difference 50,000-220,000 income 187 (out of 800) 119 (out of 800) 91 (out of 800) 74 (out of 800)



                            Table 1.  The difference in SAT scores for school districts with mean family incomes of $220,000 and $50,000.

                            In Fig. 3 is displayed the student suspension rate as a function of income.  The functional form is close to exponential which means that an extra $30,000 in income results in the halving of the suspension rate.
                            Image:NJ School Districts: SAT Scores, PARCC Scores, Suspension Rates & Family Income
                            Figure 3.  The effect of family income on suspension rate is exponential; an extra $30,000 in family income lowers the suspension rate by 50%.


                            In Table 2 is displayed the financial correlations of an SAT point.

                            Increase in tuition per year Increase in earnings per year
                            1 point increase in SAT score $36 $61



                            Table 2.  The consequence of one SAT point.




                            Discussion

                            Many people live in particular New Jersey towns for the school districts and at least some select the school district based on the SAT scores.  I find that each SAT point correlates with an increase in the mean family income of about $1,000.  On the other hand, 74 points out of 800 are presumably attributable the cost of SAT coaching.  If the most expensive SAT coaching is $7,400 this is a lot cheaper alternative at a one-time cost of $100 per SAT point.  

                            I find that college average SAT scores correlate with $61 additional yearly income per SAT point, partially offset by a tuition increase of $147 for a four year stay per SAT point, suggesting that SAT coaching is worthwhile (breakeven point occurs after four years) but a better neighborhood may not be worthwhile (Tarnow, 2015).

                            Interestingly, the tuition increase for an SAT point is suboptimal for the colleges: for every point increase in the SAT score, a college can expect an increase in the median family income of $3,000 (in NJ, it may be different in other states).  However, they only tap $147.

                            Colleges could use an “effective” SAT score with the role of family income removed but if they did, they would correspondingly compromise their financial needs.

                            Within the PARCC test I find large variations in how much the median family income influences the subject scores (Fig. 1).  The influence is the lowest for Algebra 1 and 11th grade English. The reason is unclear though I venture to guess that it has to do with a lack of parental involvement: Algebra 1 is taught both in middle school and high school and may be harder to keep track of and 11th grade English may be a topic hard to understand and/or control.  I note that New Jersey is not a state in which parents are particularly involved with the schools other than comparing school districts.

                            The suspension rate declines exponentially with median family income.  $30,000 halves the suspension rate.  The cost of moving from a “bad” neighborhood to a better one is extremely expensive.  Interestingly, teachers in the NJ school system can do this for free even though others can only do this at a substantial cost.  The freedom of teachers to move districts may also be a reason for low performing school districts staying low performing.  I suggest a solution to the problem: tax the contract benefit of moving to a richer school district at the difference in median family income between the two districts.
                            Comments Disabled

                            The Opium for the Poor Is – Opium. Providers in States with Low Income Prescribe High Levels of Opiates

                            Eugen G Tarnow  August 26 2016 11:00:10 AM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            The current opiate epidemic has many reasons: pharmaceutical marketing, naïve and/or unscrupulous doctors, availability of cheap and cleverly sold Mexican heroin and inappropriate policy changes to encourage the idea that pain was something curable with drugs that were not addictive (Quinones, 2015).

                            I find that another reason is lax supervision of Medicare providers.  I find that the Medicare opioid claims per Medicare recipient varies strongly by state.  It should not.  Medicare should have caught that but did not.  

                            Interestingly the states with the largest number of claims are the poorest - with 51% of the variance coming from the median household income.

                            Image:The Opium for the Poor Is – Opium. Providers in States with Low Income Prescribe High Levels of Opiates
                            Comments Disabled

                            NJ School Performance Reports are surprisingly interesting

                            Eugen G Tarnow  August 19 2016 04:40:48 PM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            Last February the 2014-2015 school performance reports came out.  Here is a comparison between the high schools of two neighboring towns in New Jersey, Fair Lawn and Glen Rock.

                            1.  Fair Lawn has not posted the report on its high school website for the last two years - the only link available is to the old 2012-2013 report which is announced as "now available".  Glen Rock does not post any link at all.
                            2.  In English language arts/literacy the percentage who met or exceeded expectation was 57% in Fair Lawn and 42% in Glen Rock.
                            3.  In Math the corresponding percentages were 43% (Fair Lawn) and 28% (Glen Rock).
                            4.  The participation rate was 85% in Fair Lawn and 78% in Glen Rock, presumably indicating that the true difference in English and Math scores between Fair Lawn and Glen Rock may be larger.
                            5.  Biology proficiency is 72% for Fair Lawn and 85% for Glen Rock.

                            Let's think about that for a minute.  If Fair Lawn is generally better than Glen Rock in both English and Math, why should the Biology score be better for Glen Rock?  There seems to be a problem with the Fair Lawn Biology program.

                            6.  The trend of meeting or exceeding expectations in English in Fair Lawn is downwards as students move from 9th to 11th grade: 60%, 54% and 36% in 11th grade.  Glen Rock has the same issue with a trend that goes 54%, 19% and 26%.

                            The conclusion is that there seems to be a serious problem with the high school English program in both schools.

                            7.  There is no such trend in Math, but the meeting or exceeding expectations in the Algebra 1 class of Glen Rock High School is a surprising 10%.  What is going on there?

                            8.  The average SAT score was 1602 for Fair Lawn and 1744 for Glen Rock.

                            Wait a minute.  Fair Lawn scores better in Math and English but when it comes to the SATs, Glen Rock wins hands down.  The SAT score seems to have nothing to do with the high school test scores!  Could it be that everyone in Glen Rock hires a tutor that is much better than its high school teachers?  Or do the Glen Rock students suddenly realize that they have to perform? How can Fair Lawn students copy that?
                            Comments Disabled

                            Big Data: If you can save 1% your are doing well - if you can save 30% run away! Response to Bloomberg’s Leonid Bershidsky

                            Eugen G Tarnow  June 2 2016 09:10:03 AM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            My experience in big data has taught me one piece of "big data philosophy": if you can save 1% or so, you are doing great.  If you see a way to save 30% - run away as quick as you can!

                            Before I explain the reason for this philosophy, let's use an editorial in Bloomberg by Leonid Bershidsky as background.  The title is "Big Data Is Still Only a Little Helpful and is quoting an article by Hyunyoung Choi and Hal Varian.

                            Lenoid writes that trends in Google queries can predict the number of current autosales before official numbers are made available.  In other words, because Google calculates auto query reports much faster than official reports are put together on actual sales, the Google data can be helpful.  The predictions can be improved by 5-20% according to Choi and Varian but they are looking at second order differences.  The forecast without Google may be 6.34% and with Google trends 5.66% giving an improvement of 10.6% but the actual difference is less than one percent.

                            If we consider the Choi and Varian improvement to be a 1% improvement, in my book they did a great job. People are pretty good at what they are doing and using a computer can make small improvements but it is hard to think of ways it can make a big difference on problems that have been studied many times over.

                            Or can it?

                            Sometimes it is easy to find much larger improvements using big data.  For example, I found that provider Medicare reimbursements are too high and they increase list prices by an average of 45%, that Medicare payments to hospitals for heart attacks could be lowered by 37% without effecting the quality of care.

                            Why didn't Leonid Bershidsky write about these results instead and change the title to "Big Data Can Save 40%"?

                            That is because if your big data algorithm finds such a large number it is likely it has encountered systemic corruption.

                            Finding 30% in fields such as health care is like shooting fish in a barrel.  But then run away unless you are big data's equivalent of Dirty Harry.
                            Comments Disabled

                            Who is more eloquent - conservatives or liberals? The grade level of the Facebook political discourse.

                            Eugen G Tarnow  May 18 2016 01:55:14 PM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            The Wall Street Journal recently devised a side-by-side comparison of what very conservative and very liberal users see when they go on Facebook.  

                            I thought it might be interesting to see whose discourse level was more sophisticated - conservatives or liberals?  

                            I used the comments that travel around with the news articles on all the topics categorized by the Wall Street Journal: "Hillary Clinton", "Donald Trump", "Bernie Sanders", "Barack Obama", "Guns", "Abortion", "ISIS", "Transgender" and "Facebook" and copied them into Word, replacing all names by "Jane" to remove any bias from "Donald Trump" having fewer characters than "Hillary Clinton", for example.

                            I then ran the Readability statistics and found:

                            Very conservative: 7.5

                            Very liberal: 6.6

                            using the Flesch-Kincaid Grade Level.

                            I did not check what the error bars might be (probably a full grade) but note that our high school education seems under utilized for both our very liberal and very conservative friends.

                            This entry is scored at the Flesch-Kincaid Grade Level 12.  

                            The texts used are displayed below:

                            Here's why Jane is APPEALING and ... Jane is APPALLING.
                            Brave Navy Seal DESTROYS Jane Right To Her Face.
                            This is awesome! We solute you soldier.
                            Can we get 5,000 'LIKES' for this Seal?
                            Jane's latest attack against Jane is a perfect example of why many Americans don't want her anywhere near the White House. This could be a BIG preview of Jane's strategy for the general election.
                            Will these kind of attacks help Jane defeat Jane?
                            Despite Jane's desperate attempts the facts of what really happened in the Benghazi attack are coming to light.
                            The New York Times's hit piece on Jane continues to unravel. Now Jane says she has nothing but respect for Jane and they lied about her account too.
                            ?I want to promulgate the idea that just because someone has a different opinion than yours it does not make them the enemy,? Jane said.
                            "I?ve experienced more pure, unadulterated anti-Semitism since coming out against Jane?s candidacy than at any other time in my political career. Jane supporters...greeted the birth of my second child by calling for me, my wife, and two children to be thrown into a gas chamber."
                            Want nightmares?
                            WATCH:
                            He literally lost delegates with no one else running.
                            HEY SOCIALIST Jane FANS: CHECK OUT THE SHOCKING VIDEO OF SOCIALISM IN VENEZUELA
                            This family is not fit to run a lemonade stand let alone a country.
                            MSNBC?s Nicolle Wallace argued that if Jane loses to Jane in November it will be because Jane stayed in the presidential race so long and due to his effective attacks on the Democratic front-runner.
                            Jane supporters -- are you getting this?
                            "As the young American left continues to champion the rise of socialist policies in our own country, many are left wondering if these idealistic collectivists are really prepared for the realities of socialism." -- Generation Opportunity
                            SHOTS FIRED INTO Jane NEVADA HEADQUARTERS!
                            Public schools in Jane's hometown of Chicago have just taken this transgender bathroom insanity to the next level.
                            What they're demanding students do now shows exactly how set they are on brainwashing our children that perversion is perfectly normal.
                            This is sick, and it must be stopped NOW.
                            This is not OK!
                            MUSLIMS TAKING OVER A SMALL TOWN IN TENNESSEE In Jane's Plan For A "New America" [Video]
                            Uh...
                            What treachery. It it the greatest acts of treason by a President in American history (rivaled by the Benghazi betrayal and cover-up) and yet not a critical word in the press. Scant attention is paid to Jane's towering betrayal.
                            How many of you agree she should be headed for prison instead of the White House?
                            "People who really are struggling against oppression do not resort to inventing slurs or smelting smoking guns; they have enough to deal with in the real world."
                            Who could have ever predicted this would happen?
                            Jane was molested by male migrant gang in New Year attacks. The 17-year-old and her friend described it as 'the worst night of our lives.' She said worst part was witnesses, including police, did nothing to stop it.
                            These girls need guns just like the Jews needed guns -- Germany is reverting but this time the victims are Germans.
                            NYT & The Left are desperately trying to make Jane look like a cad.
                            Why don't they turn those guns of The Lolita Express's frequent flyer and serial rapist, Bill Jane?
                            ACLU Trying to Shut Down Catholic Hospitals Because They Refuse to Do Abortions
                            Yesterday's Supreme Court ruling on President Jane's abortion-pill mandate was a major victory not only for the Little Sisters of the Poor, but also for the religious freedom of every American.
                            Read and Share as we explain the Court's ruling and why it is significant.
                            Louisiana Legislature Passes Bill to Ban Dismemberment Abortions Tearing Off Baby?s Limbs
                            It's time to end taxpayer funding for abortion providers. Click here!
                            BREAKING: South Carolina Legislature Passes Bill to Ban Abortions After 19 Weeks When Babies Feel Pain
                            ***ALERT***
                            JANE was right again.........
                            At Least Nine Americans Who Joined ISIS Were Immigrants to US!
                            CARTEL OR ISIS? TEXAS FATHER Pushes For Answers In Son's Murder: "They cut my son's head off and they took my son's head!"
                            And it's crickets from the main stream media on this horrific crime...
                            63 confirmed deaths!!
                            Here's why Jane is APPEALING and ... Jane is APPALLING.
                            These Target bosses and employees were just starting to get comfortable with the company's sick transgender bathroom policy.
                            But then they looked outside and saw who was waiting for them in the parking lot, and that changed everything.
                            Now they're as worried as they've ever been. Good. We need to keep up the pressure until they remove this perverted rule once and for all.
                            Target has committed itself to keeping its transgender bathroom policy in place, and now many of its stores are going to be facing HARSH consequences. Executives at Target never expected this to happen, and it may cause the company A LOT of damage.
                            Share this if you are boycotting Target!
                            Public schools in Jane's hometown of Chicago have just taken this transgender bathroom insanity to the next level.
                            What they're demanding students do now shows exactly how set they are on brainwashing our children that perversion is perfectly normal.
                            This is sick, and it must be stopped NOW.
                            OUR GAY PRESIDENT SPEAKS: If being transgender is now considered a human right, then where does it end? When people want to have sex with animals will that become a human right? Does the North American Man Boy Love Association qualify for protection under Jane?s madness? If ?all love is love?, as Jane and the LGBT Mafia keep telling us, then NAMBLA should absolutely come under Federal protection.The bottom of this pit runs very deep, and it ends in Hell.
                            Jane Appoints Mentally Sick Transgender Man to Lead Presidential Council
                            They probably won't let you read this, but we'll try anyway....
                            Will Facebook's Mark Zuckerberg sit down, man to man, with Milo Yiannopoulos?
                            Please take a moment and go to the website and check out ANY article as Facebook is playing games with our algorithm again. Thank you!
                            The liberal media are ignoring a MAJOR part of the Facebook scandal.
                            Facebook was recently caught red-handed suppressing conservative news and viewpoints.
                            According to this liberal at the New Republic, free speech SHOULD be stifled if it comes from conservatives. Do you agree with him?
                            Be sure to visit us at and like us at



                            Yikes.
                            How many people need to say Jane is innocent before Republicans drop it? Never mind. We know they won't.
                            Jane bested Jane in dramatic fashion.
                            HISTORY =>
                            Jane is the winner of an astonishingly close race against Jane in Kentucky, NBC News projected on Tuesday night.
                            What an idiot.
                            Just one of the many ways he respects women.
                            Let's be real. Who wants a President Jane?
                            British Prime Minister Jane brilliantly explains why they shouldn't ban Jane.
                            Video by Occupy Democrats, LIKE our page for more!
                            The author wanted one page he could send social media "trolls" who deny Jane's anti-gay, racist, sexist rhetoric ? and insist you "prove it."
                            We were only too happy to oblige.
                            "It's sort of painful to see a good person like Jane turned into such a sullen and resentful man. And doubly painful to see him take his followers down that path too."
                            Making the case for energy and enthusiasm's importance, Jane tells Californians it ain't nearly over yet.
                            "I come from the working class of this country and I'll be damned if we will allow the Republican Party, whose job is to represent the rich and the powerful, to win the votes of working-class Americans." ?Sen. Jane
                            Jane notches another victory.
                            TYT Politics Reporter Jane reported from Senator Jane Portland, Oregon headquarters on May 15th, 2016. Inside lies walls draped with incredible artwork depicting the progressive Senator and other themes of the progressive movement.
                            BREAKING ?
                            What an idiot.
                            This disproves everything Republicans have been saying about raising wages.
                            This disproves everything Republicans have been saying about raising wages.
                            Jane hates aiding the sick, apparently...
                            LIKE A BOSS! Pres. Jane Makes Overtime Pay Available To 4 Million Americans. Read more on the rules issued today by POTUS.
                            Grio fam, what are your thoughts on this?
                            I wonder how he's going to explain that mustache in prison? He had lots guns and ammo on him, too.
                            Doesn't this sound kind of familiar? Isn't this the same fear-mongering the GOP tried when President Jane ran?
                            Something is terribly wrong with America, babies, and guns. Don't know what the solution is, but this is officially way past insane.
                            What is he smoking?! I guess he was just in Washington state.
                            AC/DC have released a video showing rehearsal footage ahead of their live debut with temporary frontman Guns N' Roses' Jane. Watch below.
                            ?It wasn?t abortion that birthed the religious right. It was good old white Nativism and anti-government anger when the IRS challenged evangelicals? god given right to go to school without black people.?
                            He is totally busted.
                            For over 30 years, Jane has been dedicated to keeping the clinic doors open to ensure women of Alabama have access to reproductive healthcare that includes abortion care services. Last year she was one of the recipients of the ACLU?s the Jane Medal of Liberty because she is a dedicated nurse, mother, provider and hero.
                            Alabama Governor Jane has signed into law a measure banning abortion clinics from operating within 2,000 feet of a K-8 public school.
                            Isn?t it ironic: tough abortion restrictions do not lead to fewer abortions? Restrictions do not prevent abortion, but they do make it less safe. As abortion restrictions sweep across the country, new reports show that such restrictions don?t actually decrease the abortion rate.
                            This week ISIS has stepped up violent attacks in Iraq ? but you might have missed it.
                            WARNING: Graphic footage.
                            This is proof that Republicans don?t care about ISIS unless they?re using it to scare up votes.
                            These are outright lies. And Jane just keeps repeating them.
                            What's the difference between Jane and ISIS? It's starting to sound like not much.
                            The war against ISIS is being conveyed to us as if we're on Snapchat. Important things happen, they appear in front of us and then... POOF!... they're gone. But just because we now have terrifyingly short memories does not, however, mean that critical things did not happen.
                            A stark contrast on LGBTQ issues.
                            Being transgender is not a mental disorder ? so why is it still classified as one?
                            This man doesn?t belong on a public platform. He belongs in an institution.
                            She donated her hair to kids with cancer, then she met a bigot in a bathroom.
                            Read the whole story HERE
                            MISSING: 19-year-old Jane was last seen leaving a dorm party on campus. She was on a full scholarship at Columbia University in the City of New York.
                            The student's mobile phone & bank account numbers have been changed. Her Facebook page has been deactivated as well. DETAILS ---
                            Jesus is going to be so disappointed in him.
                            Um...
                            Last November, she posted a message on Facebook saying she had taken an overdose.
                            This is the result of anti-LGBT hate speech..


                            Comments Disabled

                            The 2008 mortgage crisis in clusters

                            Eugen G Tarnow  May 16 2016 09:20:13 AM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            I noted in a previous posting Fannie Mae mortgages & credit scores that it appeared that the credit score distribution for the 2007 Fannie Mae loans had two peaks.  

                            One can, in fact, see clustering in the credit scores.  Interestingly, there are also two clusters in 2005, 2003 and even 2001!  

                            What do we know about those clusters?

                            The most important definer of the clusters is the average credit score of the borrower:

                            Image:The 2008 mortgage crisis in clusters

                            Notice how the clusters are far apart before the mortgage crisis.  The largest difference occurs in 2007 in which the difference is 102 points.  
                            Yet the loan amounts to these two clusters are almost the same:

                            Image:The 2008 mortgage crisis in clusters

                            The smallest difference in loan amounts occurs in 2003 in which it is 1.8%.  How can any borrower decide to loan the same amount to two populations that differ by 100 points in the credit score?

                            Before the mortgage crisis the number of borrowers in the high score cluster goes down to 56% and after the mortgage crisis it goes up to 72%:

                            Image:The 2008 mortgage crisis in clusters

                            Beyond the credit score, the clusters also differ by the proportion single borrowers:

                            Image:The 2008 mortgage crisis in clusters

                            In the low rated cluster it starts out at 5% higher in 2001 but it keeps increasing even through the mortgage crisis.  In 2013 the difference was 13%:

                            Image:The 2008 mortgage crisis in clusters

                            The interest rates are barely different for the two groups:

                            Image:The 2008 mortgage crisis in clusters

                            From the data available to me I am not able to discern the composition of these clusters.  

                            But it is curious that they exist at all.
                            Comments Disabled

                            The Lift of An Alzheimer’s Memory Test

                            Eugen G Tarnow  May 2 2016 04:58:45 PM
                            The proportion of old people is increasing and with that increase follows an increase in particular types of disease, one of which is Alzheimer’s disease.  If Alzheimer’s disease is not an example of Gompertz’ law (an exponential death rate purely due to age) it might be possible to design drugs that may slow down or even reverse the disease progress.

                            In order to put those drugs to clinical trials, symptoms of Alzheimer’s disease have to be easily measurable.  The “gold standard” of Alzheimer’s symptoms, unfortunately, are the results of an autopsy.  Thus “silver” standards have to be used.

                            One of the silver standard symptom categories are memory failures.  Alzheimer’s disease attacks the brain and causes damage to the memory.  There is some reason to believe that the damage follows a particular path in the brain and the memory failures would then follow a well-defined path.  It is believed, for example, that the diagnosis of “mild cognitive impairment” represents early Alzheimer’s disease.

                            While it is unlikely that damage to long term memory could be reversed by drugs (even if the brain tissue comes back the information is gone), damage to long term memory may be slowed and damage to short term memory may even be reversible.

                            But what is short term memory?  Despite what is commonly believed this question is not settled.  Accordingly, there are many Alzheimer’s short term memory tests.

                            One of the memory tests most sensitive to Alzheimer’s disease is “free recall” in which a set of words are studied and then repeated.  Unfortunately, words can be “chunked” together, creating larger meaningful units that each are as easily remembered as a single word.  Some of us can chunk words more easily than others so that a word free recall test of ten words can appear to some as just five “chunks”.  Thus test results cannot be compared between individuals and instead would require multiple tests over time to detect individual changes.  

                            We at Avalon Analytics wanted to see whether an extremely well defined memory test, we named TUT (Tarnow, 2013), with items that lack meaningful connections would be sensitive to Alzheimer’s disease.  The test items were three double digit integers (the capacity of “working memory” is about 3), the particular numbers selected to make chunking very difficult.  Since the test items were common we thought that they may not be sensitive to education or language.  To lower the statistical noise level the TUT test consisted of three sets of three numbers each.

                            We were able to work with the Alzheimer’s Disease Research Center at Icahn School of Medicine at Mount Sinai (ADRC ISMMS).  The 132 participants (Table 1) were part of a clinical study for aging and dementia. Inclusion criteria included 65 years of age or older, primarily English/Chinese/Spanish speaking, visual and auditory acuity adequate for cognitive testing, willingness to participate in all clinical assessment, and having a study partner available as an informant.  Three records indicating dementia due to oncology treatment and vascular dementia were discarded as was a record of a subject who refused to recall more than one number.

                            DX
                            Frequency
                            Percent
                            Valid Percent
                            Cumulative Percent
                            Valid Normal
                            83
                            62.9
                            63.8
                            63.8
                            MCI
                            37
                            28.0
                            28.5
                            92.3
                            AD
                            10
                            7.6
                            7.7
                            100.0
                            Total
                            130
                            98.5
                            100.0
                            Missing System
                            2
                            1.5
                            Total
                            132
                            100.0



                            Table 1.  Breakdown of study subjects.

                            We found that the participants remembered on average 2.6 items, independent of language (English, Cantonese, Mandarin and Spanish), independent of gender, age and education.

                            The participants were diagnosed as normal, Alzheimer’s disease, and two types of MCI (“mild cognitive impairment”).

                            The test detected participants with Alzheimer’s disease (remembered an average of 1.9 items versus 2.6 for normal participants) but was insensitive to participants with MCI (remembered an average of 2.5 items).  The standard deviation for both Alzheimer’s disease and normal subjects were 0.6 items.  The probability of a subject in the test population having Alzheimer’s disease as a function of the number of items remembered is displayed in Figure 1.

                            The lift is calculated by taking the ratio of the highest target response of 40% from Fig. 1 and the average response (the prevalence of Alzheimer’s disease in the sample) of 7.6%.  This gives a lift of 4 at 20% (the lift below 1 is 0!).

                            If we search for a meaning of the result of our very simple memory test it is quite complex.  One of the participants with Alzheimer’s disease had a perfect score on our test.  Thus the loss of working memory capacity for double digits is a common but not necessary condition for Alzheimer’s disease (clinical diagnosis, not autopsy).  Perhaps there are separate working memories for various item categories?  Or is the memory system with the worst symptoms geographically close in the brain to where the working memory for double digits is?  In addition, the participants with the lowest working memory capacity for double digits did not have dementia.  How is it possible to function with such an impaired memory?  Perhaps double digits are just not that important?

                            Time will tell.

                            Image:The Lift of An Alzheimer’s Memory Test
                            Fig. 1. Probability that a subject coming to the clinic will have AD as a function of the TUT 3 item recall.

                            Bibliography:

                            Tarnow, E. (2013). U.S. Patent Application No. 14/066,195.

                            Tarnow, E. (2016). Preliminary Evidence -- Diagnosed Alzheimer's Disease but Not MCI Affects Working Memory Capacity - 0.7 of 2.7 Memory Slots is Lost. arXiv preprint arXiv: 1603.07759.

                            Biography:

                            Eugen Tarnow is a Director of Data Science at Avalon Analytics (see http://avalonanalytics.com ).  He has a PhD in physics from MIT and have published research in a variety of fields including surveys, marketing, air plane cockpit behavior, and human memory.

                            He can be reached at etarnow@avabiz.com
                            Comments Disabled

                            Employers are unhappy with your education - so take the PARCC!

                            Eugen G Tarnow  April 7 2016 09:56:09 AM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            I just went to hear a presentation about why my child should be participating in the PARCC testing by Donna Egreczky of the NJ Chamber of Commerce.

                            Here data included:
                            • 49% of employers dissatisfied with high school students entering workforce
                            • 76% of college instructors report students were not prepared at all for college expectations
                            • 76% of college students report SOME gap in overall preparedness.  47% report LARGE gaps.
                            • Armed forces reject 75% of applicants
                            • College degree not predictive of career success
                            • High grades in high school not predictive of college degree attainment

                            This is a fascinating "big data" analytics problem.  

                            And the devil is in the details.

                            I do not think participating in the PARCC testing is the "nugget" that will lead to improvements in the data she presented.

                            Rather, it is to do a knowledge census of the working population every ten years, see what of this can be taught in grade school and stop the mandatory education in 8th grade.  After all, the working population, on average, probably functions on the 8th grade level just fine.  And the school curriculum set after World War II is not relevant anymore.  And neither high school nor college teach "critical thinking" - perhaps it cannot be taught at all.
                             

                            Comments Disabled

                            Congratulations to Rainer Weiss for discovering gravity waves!

                            Eugen G Tarnow  February 11 2016 01:18:49 PM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            When I was an undergraduate at MIT, it was hard to get into a research group because as an undergraduate you are pretty useless!  

                            The first group that accepted me was the one headed by Rainer Weiss, a professor that reminded me of Groucho Marx.  Instead of a cigar he had a pipe.  

                            Rainer Weiss really cared about his students.  I found out that I was not suited for experimental physics after almost blowing up a container of liquid helium  a couple of times.  He became my advisor nevertheless and in the end I got to do some theoretical research and went on to do a PhD in physics.

                            Back then Rainer Weiss proposed a way to detect gravity waves and kept trying to get the project going.  This was 35 years ago.  He did get it going and it was just announced that he succeeded.

                            I can't think of anyone more deserving!
                            Comments Disabled

                            Estimating the cost of your hospital

                            Eugen G Tarnow  February 9 2016 03:28:24 PM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            Hospital charges can vary widely - the same visit can result in different number of charged items and different charges for each item.

                            So we created CostOfHospitals.com,  a website which estimates hospital charges (using our own algorithm).  We do not guarantee results but hope it might be helpful for both those with insurance (some hospitals may charge more items per visit) as well as those without insurance.
                            Comments Disabled

                            AVVO lawyer ratings are statistically unusual

                            Eugen G Tarnow  February 9 2016 11:23:16 AM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com


                            Internet ratings are a fascinating topic (I have covered drug ratings before - see Internet drug ratings: AskAPatient.com & WebMD correlate well but Drugs.com does not).  Yelp has recently been in the news with its ratings.  AVVO rates lawyers in a controversial way and has been taken to court but they won.

                            I have been interested in AVVO ratings because it is in AVVO's interest to rate lawyers who do not use them artificially low and then increase their ratings when the lawyers "claim their profiles," i.e. start to work with AVVO and perhaps pay for their listings. If this is part of their business model one would find statistical properties of the ratings that deviate from what would be the norm.

                            This entry shows that there is an unusual distribution of AVVO lawyer ratings.  

                            Typically, a rating scheme would yield a normal curve - i.e. one peak in the middle and slow decrease from that in both directions.  I did a Google search on AVVO ratings today, and found the following distribution for ratings 6.1 and above (the AVVO ratings go up to 10.  I avoided the ratings below 5 because in my Google search the consumer ratings (5 and below) interfered with the AVVO ratings):


                            Image:AVVO lawyer ratings are statistically unusual
                            Figure 1.  AVVO ratings of lawyers.


                            There are very large peaks at 6.5, 6.7 and 8 with hardly any lawyers in-between.  The coveted 10 out of 10 spot has a peak but there are almost no ratings from 8.9 to 9.9.

                            AVVO claims their ratings represent the quality of lawyers according to the following schema:

                            10.0 - 9.0 Superb
                            8.9 - 8.0 Excellent
                            7.9 - 7.0 Very Good
                            6.9 - 6.0 Good

                            If that is true the ratings should obey a normal distribution.  There appears to be many lawyers that are "good" and many that are on the bottom of the "excellent" category but hardly anyone who belong in the "very good" category.  And in the superb category, basically everyone is at the top of the category.  All this makes the AVVO ratings statistically highly unlikely.  Or there is something wrong with my Google search terms.

                            If the business model is based on giving non-cooperating lawyers low ratings and cooperating lawyers high ratings one would expect that higher peaks would belong to cooperating lawyers while lower peaks would belong to non-cooperating lawyers.

                            Indeed, the first five entries for lawyers rated 8.0 had pictures and their profiles had evidently been claimed by the lawyers.  In the first five entries for lawyers rated 6.7, 2 lawyers had not claimed their profiles.  In the first five entries for lawyers rated 6.5, however, all the profiles had been claimed.

                            I decided to email one of the 6.5 rated lawyers to find out why he is rated low and found that the AVVO site immediately suggested other lawyers to write to as well.  I decided to try the same with a 10 rated lawyer and no other lawyers were suggested.  That suggests that lawyers with 10 ratings is getting better AVVO services than lawyers with 6.5 ratings.  That might mean that lawyers with 10 ratings pay AVVO more.

                            This may be a measure of the number of high quality customer AVVO has.  If so this number is 5800.

                            My investigation only shows that the rating distribution is likely very unusual, not that there is a direct link between payment and ratings.  

                            Thus it is possible to rate the AVVO rating on statistical plausibility.  We give them 0.5
                            stars out of a possible 5.
                             
                            Comments Disabled

                            Comparing holy texts? How the Bible looks different in two different translations

                            Eugen G Tarnow  February 4 2016 09:55:22 AM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            It is tempting to use Natural Language Processing to compare holy texts across religions as was recently done by a colleague of Avalon Analytics.  But there is a need to be careful because there is much riding on the details.

                            Below I show you what a word frequency comparison looks like of the SAME text (Bible from http://unbound.biola.edu/index.cfm?method=downloads.showDownloadMain ) but in different versions.  The top panel displays the "Basic English" version and the bottom panel displays the "American Standard" version.  To make the comparison nothing has been done to the text except removal of "stopwords" (words that are too common to convey understanding.  List is taken from the NLTK library).

                            I note two severe discrepancies.  In one version "shall" is extremely important but in the other it is nowhere to be found; in one version "give" is important but in the other version - it can't be found!

                            Image:Comparing holy texts? How the Bible looks different in two different translations
                            Image:Comparing holy texts? How the Bible looks different in two different translations
                            Figure 1.  Word frequency plots of the Bible.  Top panel is the Basic English version, the bottom panel displays the American Standard version.

                            Comments Disabled

                            Medical device deaths do not attract attorneys?

                            Eugen G Tarnow  February 3 2016 12:20:23 PM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            There is a new tool for class action attorneys specializing in medical device suits: http://fdafeed.com .  That site displays manufacturers together with the number of death reports and attorneys mentioned.  The data is taken from an FDA database.  

                            Interestingly there is not much of a correlation between the number of death reports and number of attorneys (see Figure 1).  Is it because most deaths do not lead to possible legal claims or is it because attorneys are not yet using big data?


                            Image:Medical device deaths do not attract attorneys?
                            Figure 1.  Number of death reports versus number of attorneys for medical device adverse events.

                            Comments Disabled

                            Pew Research creates something out of nothing: fooling Financial Times and Naked Capitalism with a particular definition of "Middle Class"

                            Eugen G Tarnow  December 14 2015 09:14:05 AM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            D'Israeli, former Prime Minister of England, coined the phrase "Lies, damned lies and statistics".  And Pew Research is a good example, with Financial Times and Naked Capitalism on the band wagon.

                            According to CNN, there are five ways to define "Middle Class": income, wealth, consumption, aspiration and demographics.

                            Pew Research decided to use income as its definition.  Within income, CNN includes several ways to define middle class: middle fifth, middle three fifths and Pew's definition ranging from 2/3 to 2 times the median income.

                            Pew's headline screams out: "The American Middle Class Is Losing Ground" leaking members to those "in economic tiers above and below it".  The Financial Times writes "America’s Middle-class Meltdown: Core shrinks to half of US homes" and Naked Capitalism concludes: "Demise of the US Middle Class Now Official".

                            If Pew would have chosen a different income based definition of the middle class: the middle fifth or the middle three fifth in income, they would have had no news...

                            A comment on Naked Capitalism went unpublished despite its motto of "fearless commentary" - they apparently feared this one.
                            Comments Disabled

                            Pew, Financial Times & Naked Capitalism get fooled by statistics - claim the middle class is disappearing is simply a shift to higher incomes

                            Eugen G Tarnow  December 11 2015 02:31:45 PM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            Sometimes I fall off my chair when I see statistical incompetence.  This time Pew Research, Financial Times & Naked Capitalism all fell victims to their own stupidity.

                            The claim is that over time the middle class is disappearing and that this is shown by data from the Current Population Survey.  These two graphs are shown in Naked Capitalism (
                            http://www.nakedcapitalism.com/2015/12/demise-of-the-us-middle-class-now-official.html ):

                            Image:Pew, Financial Times & Naked Capitalism get fooled by statistics - claim the middle class is disappearing is simply a shift to higher incomes

                            Image:Pew, Financial Times & Naked Capitalism get fooled by statistics - claim the middle class is disappearing is simply a shift to higher incomes
                            Fig. 1.  House hold income distributions in 1971 and in 2015 (or should it be 2014?) from Naked Capitalism.

                            And the Naked Capitalism commentary goes "The Financial Times headline is uncharacteristically dramatic: America’s Middle Class Meltdown: core shrinks to half of US homes. And I find their infographic that charts the the shrinkage of the middle class cohort over time to be more informative than the Pew charts that presented the same information; we are partially replicating it by showing the starting and end shots:"

                            Really?  Incomes obey what is called a "lognormal" distribution because they are always positive.  Below I show you what happens when the average of a log normal distribution increases (yes, people make more money over time!).  The blue distribution has an average "income" of 1 and the red distribution has 60% higher average "income".  Indeed, it looks like the "middle class" disappears but it is just a function of how a lognormal distribution changes its look when the average increases.  And the +200k spike increase in the graphs of naked capitalism is there too.  Of course, if the incomes increase, then the spike, which is the sum of all incomes over the scale of the graph, has to go up a lot...

                            Image:Pew, Financial Times & Naked Capitalism get fooled by statistics - claim the middle class is disappearing is simply a shift to higher incomes
                            Fig. 2.  Changes in the "look" of a lognormal distribution as the average increases.  At 10, the point represents the sum of all values above 10.

                            A comment on Naked Capitalism went unpublished despite its motto of "fearless commentary" - they apparently feared this one.
                            Comments Disabled

                            Sales people do not care about analytics!?

                            Eugen G Tarnow  December 3 2015 08:47:16 AM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            One would think the people closest to the money in an organization would care the most about using analytics.  Data from application sales at Salesforce.com seem to invalidate this assumption.

                            While the actual sales of AppExchange apps is not public information, the popularity and number of ratings are displayed.

                            The average number of ratings of the overall top 20 pay applications is 584 while the corresponding number for analytics apps is only 14, 2.3% of the total.

                            Image:Sales people do not care about analytics!?

                            The question is why?  Is it that sales people do not want to be supervised or that their task is so socially complex that a computer app cannot be helpful?
                            Comments Disabled

                            An Inexpensive Way to Measure Loci of Control: Counting Will, Must & Should

                            Eugen G Tarnow  November 4 2015 05:19:40 PM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            One of the famous personality tests in psychology, the Rorschach ink blot test, yields a lot of interesting data.  One of them may surprise: the locus of control.  The locus of control is a way to say whether you feel that you control your own destiny or that you feel that your destiny is controlled by others.  

                            The cost of a Rorschach ink blot test is in the thousands of dollars.  Might there be a cheaper way to do it?

                            I suggest that one might be able to estimate the locus of control from a person's writings. Here is a pilot study that focuses on the use of three words: "will", "must" and "should".  "Will" corresponds to an internal locus of control and "must" corresponds to an external locus of control.  "Should" expands the definition of the locus of control to include a vector that can point either towards or from the person.  Of course, the meanings of the three words overlap to some extent but for now, let's go with these assumptions

                            So where are authors' loci of control in the common Gutenberg library?  In Fig. 1 we find that the authors with the least number of "wills" are Austen, Blake and Carroll.  The book with the most "wills" is the Bible.  The author with the most "shoulds" is Blake.  

                            If we take the difference between "wills" and "musts" to find the book with the strongest internal locus of control (see Fig. 2) it is the Bible. This makes sense since it is written primarily from the point of view of God.  Why Austen and Carroll are the authors with the strongest external loci of control can be a college course in itself, but seems plausible.

                            To summarize, I have presented a new way to measure the locus of control by counting the relative frequency of the words "will", "must", and "should".


                            Image:An Inexpensive Way to Measure Loci of Control: Counting Will, Must & Should
                            Fig. 1. The number of "wills", "musts" and "shoulds" in the Gutenberg library.


                            Image:An Inexpensive Way to Measure Loci of Control: Counting Will, Must & Should
                            Fig. 2. The number of "musts" subtracted from the number of "wills" in the Gutenberg library.

                            Comments Disabled

                              Dr. House’s Vicodin and Sheriff Longmire’s Zoloft criminals - how well did pharma do?

                              Eugen G Tarnow  November 3 2015 10:29:56 AM
                              By Eugen Tarnow, Ph.D.
                              Avalon Business Systems, Inc.
                              http://AvalonAnalytics.com

                              A very famous social psychologist, Stanley Milgram, once tried to find out whether violence on TV translates into violence in society.  His findings were that the crime rate did not go up after a crime was shown on TV, BUT that the type of crime tended to imitate the TV show.

                              Recently it was found that the number one drug for Medicare patients is Vicodin, the drug that gets Dr. House hooked on TV.  Did the TV show influence the number of prescriptions of vicodin?  If so, in which direction?

                              We don't have the resources to do this project, but someone ought to do it.  Help has been offered on youtube: https://www.youtube.com/watch?v=PSaasBjeJek

                              One would need to get the daily prescription data from IMS and then the show data (from Nielsen?).

                              Once that is done one can turn one's attention to the Longmire show's Zoloft criminals - who desperately want to make Zoloft available to prevent suicide of veterans (not much evidence that would work).  Did the Zoloft prescriptions go up?
                              Comments Disabled

                              The Death Spiral of health insurance

                              Eugen G Tarnow  November 2 2015 12:50:15 PM
                              By Eugen Tarnow, Ph.D.
                              Avalon Business Systems, Inc.
                              http://AvalonAnalytics.com

                              Ever heard of the "death spiral" of health insurance?  Well, here it how it works.  

                              Health insurance claims are dominated by the most expensive participants - see the blue line in Figure 1.  The orange line shows the total collected for the various members if the revenue is 20% over cost.
                              Image:The Death Spiral of health insurance
                              Fig.1. Total cost of health plan members starting from the most costly (blue line).  

                              The costliest 1% represent 40% of the total cost, the costliest 10% represent 80% of the cost.  

                              Income (orange line) crosses the cost curve near the full membership.





                              When is the break-even point?  This is better illustrated in Fig. 2.  The cost function (blue line) crosses the revenue line (orange line) near full membership.  If the revenues are 20% over cost, then the break-even point is at 83% of membership.  If the revenues are X over cost then the break-even point is about 1-X.  

                              Image:The Death Spiral of health insurance
                              Fig.2. Total cost of health plan members starting from the most costly (blue line) and the corresponding income (orange line).  

                              If the revenue is 20% higher than the cost, then the break-even point is roughly at 83%.  

                              If the revenue is 1+X of the cost then the break-even point is at 1/(1+X) which is roughly 1-X for small X.






                              Of course, the break-even point for the policy holders is much lower than that.  As shown in Fig. 3, it is about 13%.  Thus 87% of plan members lose money by being members.  Since health insurance costs are so large, it is tempting to forgo it.  If the 5% healthiest members leave the plan, the insurance company has to increase the price for the remaining members by 5%.  If then the next healthiest 5% balk at the cost increase, the new price goes up another 5% and so forth. At the same time the plan members get sicker with time which also increases costs.

                              That is the "death spiral" of health insurance.  It does not occur for other insurance types, presumably because the associated costs are so much smaller:  insuring one's house is only about $500 per year while health insurance for a family is $20-$30,000 per year.  

                              It would seem that one way to remove the death spiral would be to only insure against catastrophic costs. However, ObamaCare does not allow this for anyone but the poorest members and, presumably, health insurance companies would be shrinking without the gigantic premiums and health care providers would get many fewer customers if regular care would be paid by the patient instead of a third party.
                              Image:The Death Spiral of health insurance
                              Fig.3. Break-even point for members occurs at a low 13th percentile of the costliest members.





                              We at Avalon can assist with your statistics needs.  We use SPSS Modeler, Python and R to get the results.
                              Call us at 646 9825 9080  x1114 to get a quote.
                              Comments Disabled

                              Estimating the cost of your doctor

                              Eugen G Tarnow  October 27 2015 09:35:47 AM
                              By Eugen Tarnow, Ph.D.
                              Avalon Business Systems, Inc.
                              http://AvalonAnalytics.com

                              Doctor charges can vary widely - the same visit can result in different number of charged items and different charges for each item.

                              So we created CostOfDoctors.com,  a website which estimates doctor charges (using our own algorithm).  We do not guarantee results but hope it might be helpful for both those with insurance (some doctors may charge more items per visit) as well as those without insurance.
                              Comments Disabled

                              College Data: An additional $1 tuition gives you $0.45 extra salary six years after college enrollment

                              Eugen G Tarnow  October 6 2015 11:02:53 AM
                              By Eugen Tarnow, Ph.D.
                              Avalon Business Systems, Inc.
                              http://AvalonAnalytics.com

                              From the new college attendance data released by the Obama administration, one finds that earnings six years after enrollment are weakly but positively correlated with the cost of attendance.  For every $1 in extra tuition, the yearly salary increases by $0.45 per year.  

                              Is this enough to justify an expensive school?  If the college education took 4 years, it would take another 9 years to pay off the additional cost.  Is it worth it - you decide!  

                              Image:College Data: An additional $1 tuition gives you $0.45 extra salary six years after college enrollment



                              We at Avalon can assist with your statistics needs.  We use SPSS Modeler, Python and R to get your results.

                              Call us at 646 9825 9080  x1114 to get a quote.

                              Comments Disabled

                              How Avvo games the Google search engine

                              Eugen G Tarnow  October 6 2015 10:42:39 AM
                              By Eugen Tarnow, Ph.D.
                              Avalon Business Systems, Inc.
                              http://AvalonAnalytics.com

                              Those who pay for SEO can get their monies worth.  One company that thrives on using SEO is Avvo.  If you look up your favorite lawyer, chances are that an Avvo "rating" will show up nearby.  This "rating", perhaps without published reviews, will often be close to 6.2, not low enough to get the lawyer upset but not high enough that it can be ignored, perhaps to encourage the lawyer to sign up for the Avvo service.  

                              How is Avvo able to get so close to the real lawyer entries?  If one clicks on the Avvo link one finds 5 mentions of the lawyer name.  If one then looks at the page "source" the lawyer name is mentioned no fewer than 30 times! Google perhaps mistakenly thinks the information is important and there you have it.

                              Avvo has a sense of humor.  They write:

                              "Much as Google doesn’t share its search engine secrets, we don’t share the inner-workings of the Avvo Rating so we can maintain the neutrality and integrity of the rating."


                              Interestingly, lawyer marketing services seem to be closely aligned with Avvo.  There are not too many sites where you see a straightforward assessment of Avvo.  Here is one: http://www.abajournal.com/news/article/another_lawyer_sues_avvo_rating_site_claims_its_practices_are_beyond_unfair/

                              We at Avalon can assist with your statistics needs.  We use SPSS Modeler, Python and R to get your results.

                              Call us at 646 9825 9080  x1114 to get a quote.

                              Comments Disabled

                              College Data: Admitting higher SAT scores yields higher eventual earnings, especially for the SAT math score

                              Eugen G Tarnow  September 22 2015 12:00:17 PM
                              By Eugen Tarnow, Ph.D.
                              Avalon Business Systems, Inc.
                              http://AvalonAnalytics.com

                              One can find lots of interesting things in the new college dataset.  For example, how does the college average SAT score impact eventual earnings?  For every average point on the SAT verbal test, the income increases by $55 per year (see Fig. 1).  100 points, $5,500 per year.  The relationship is even more important for the SAT math score: 100 points, $6,900 per year (see Fig. 2).  So if colleges want better earning alumnae, they should admit higher SAT scoring students.  And the math score is more important than the verbal score.

                              But in general, colleges tend to not care.  In fact, the average SAT math score is skewed towards low scores (see Fig. 3).  Perhaps a current student is more important to the college income than the average alumnus income.

                              Image:College Data: Admitting higher SAT scores yields higher eventual earnings, especially for the SAT math score
                              Fig. 1.  Mean earnings six year after initial enrollment as a function of the average SAT verbal score.


                              Image:College Data: Admitting higher SAT scores yields higher eventual earnings, especially for the SAT math score
                              Fig. 2.  Mean earnings six year after initial enrollment as a function of the average SAT math score.


                              Image:College Data: Admitting higher SAT scores yields higher eventual earnings, especially for the SAT math score
                              Fig. 3.  Distribution of average SAT math scores of admitted students.  The distribution is skewed towards low math scores.



                              We at Avalon can assist with your statistics needs.  We use SPSS Modeler, Python and R to get your results.

                              Call us at 646 9825 9080  x1114 to get a quote.

                              Comments Disabled

                              Fannie Mae mortgages and credit scores

                              Eugen G Tarnow  September 16 2015 10:25:47 AM
                              By Eugen Tarnow, Ph.D.
                              Avalon Business Systems, Inc.
                              http://AvalonAnalytics.com

                              Ever wondered how your credit score affects the interest rate?  Well, in 2013 the interest rate decreased on average 0.2% for every 100 point increase in credit score.  

                              Nevertheless, the credit score only amounted to 7% of the variance.

                              Image:Fannie Mae mortgages and credit scores
                              Fig. 1. Interest as a function of borrower's credit score.


                              Mortgages typically go to partners with very small differences in credit score (see Fig. 2), the standard deviation is only 30 points, and there are almost no mortgages to partners with a credit score difference of 100 or more.  
                              Image:Fannie Mae mortgages and credit scores
                              Fig. 2.  Distribution of differences in credit scores for borrower and co-borrower


                              This suggests four explanations: that we marry those with the same credit score, that lenders do not lend if the scores are too different, that the credit score process artificially forces the scores to be similar, or that the mortgage process tends to force credit scores to become similar over time. The difference in credit scores is somewhat larger for first time home buyers (standard deviation is 40 points instead of 30 points for all mortgages, the overall credit score is 15 points lower per borrower), but that could does not exclude any of the four explanations (we may divorce those with different credit scores, the lenders are encouraged to give loans to first time home buyers, etc.), see Fig. 3.
                              Image:Fannie Mae mortgages and credit scores
                              Fig. 3.  Distribution of differences in credit scores for borrower and co-borrower, first-time home buyers only.


                              And there are very few mortgages in which at least one credit score is 600 or below.


                              Image:Fannie Mae mortgages and credit scores
                              Fig. 4.  Frequency of mortgages as a function of credit score.


                              And one might ask - how did the distribution of credit scores change from 2000 to 2007 to 2013?  In Fig. 5 we see that the distribution became bimodal in 2007!

                              Image:Fannie Mae mortgages and credit scores
                              Image:Fannie Mae mortgages and credit scores
                              Image:Fannie Mae mortgages and credit scores
                              Fig. 5.  Number of mortgages as a function of credit score during the years 2000 (top), 2007 (middle) and 2013 (bottom).  Notice the appearance of two peaks during the mortgage bubble in 2007, almost as if 700 was the new 800.


                              In 2007 the total credit score became the same for first time home buyers as for everyone else. The standard deviation of the difference in credit scored increased from 40 to 50 for first time home buyers and from 30 to 40 for everyone else.  If we divide the borrowers into the two peaks of Fig. 5 (middle panel), the standard deviation of the difference in mortgage score for the top group (borrower credit score above 725) is 35 and for the bottom peak it is 50.  Thus relaxing the borrowing conditions allows the difference in credit scores to increase.

                              We at Avalon can assist with your statistics needs.  We use SPSS Modeler, Python and R to get your results.

                              Call us at 646 9825 9080  x1114 to get a quote.

                              Comments Disabled

                              Think you have a good mortgage rate - think again!

                              Eugen G Tarnow  September 15 2015 04:08:06 PM
                              By Eugen Tarnow, Ph.D.
                              Avalon Business Systems, Inc.
                              http://AvalonAnalytics.com

                              Fannie Mae publishes data on the mortgages they get. There is a delay of more than a year so the latest complete file is from 2013.

                              This is what the distribution of a 1% sample of thirty year mortgage rates looks like:

                              Image:Think you have a good mortgage rate - think again!
                              The most common mortgage rate is 3.68% but there are people who were able to get a 3.06% rate!

                              Think you got a good deal?  Think again! (Unless, of course, the low interest loans had points paid up-front)

                              We at Avalon can assist with your statistics needs.  We use SPSS Modeler, Python and R to get the results.

                              Call us at 646 9825 9080  x1114 to get a quote.

                              Comments Disabled

                              An unpopular drug

                              Eugen Tarnow  September 4 2015 12:39:56 PM
                              By Eugen Tarnow, Ph.D.
                              Avalon Business Systems, Inc.
                              http://AvalonAnalytics.com

                              There are some drugs that are very unpopular with patients. "X" is one of them.  Here are its ratings:

                              Image:An unpopular drug
                              Figure 1.  Number of ratings as a function of rating.


                              One might wonder whether there is anything redeeming about the drug?  For example, can it be prescribed under different circumstances to make it more palatable?

                              Image:An unpopular drug
                              Figure 2. Number of ratings as a function of condition for which it was prescribed.


                              Does not look like it can!  Since 78% of patients rates the drug as a complete flop, there is not even any room left over even for the placebo effect...

                              What is its most common side-effect?  Varieties of pain!

                              Image:An unpopular drug
                              Figure 3.  Words preceding "pain" in patient reviews.



                              We at Avalon can assist with your statistics needs.  We use SPSS Modeler, Python and R to get the results.

                              Call us at 646 9825 9080  x1114 to get a quote.

                                Changing the survey question, getting a different result but no correction issued by Quinnipiac University

                                Eugen G Tarnow  August 29 2015 11:17:16 AM
                                By Eugen Tarnow, Ph.D.
                                Avalon Business Systems, Inc.
                                http://AvalonAnalytics.com

                                We are survey experts and once in a while we notice disturbing survey results.

                                These are the results in question:

                                April 27, 2015 - American Voters Back Iran Deal By Wide Margin, Quinnipiac University National Poll Finds


                                and

                                August 3, 2015 - American Voters Oppose Iran Deal 2-1, Quinnipiac University National Poll Finds


                                Really?  How is it possible that American voters changed their minds so quickly and fundamentally?  

                                It turns out they may not have, but the questions asked were different.  In particular, the question asked in the April poll was a leading question:

                                "55. As you may know a preliminary agreement was reached in which the United States and other countries would lift major economic sanctions against Iran, in exchange for Iran restricting its nuclear program in a way that makes it harder for it to produce nuclear weapons. Do you support or oppose this agreement?"

                                The meaning of the Iran deal was laid out tor the voters, instead of letting the voters decide what it meant.

                                In the August poll the question was neutral:

                                "67. Do you support or oppose the nuclear deal with Iran?"

                                Quinnipiac University did not issue a retraction of the earlier survey, nor did they discuss how they changed the question for the August survey.  

                                A similarly leading question on Iran was used in a poll commissioned from Social Science Research Solutions by the LA Jewish Journal.  They asked:

                                "As you know, an agreement was reached in which the United States and other countries would lift major economic sanctions against Iran, in exchange for Iran restricting its nuclear program in a way that makes it harder for it to produce nuclear weapons. Do you support or oppose this agreement, or don’t know enough to say?"

                                They got a result very similar to the April Quinnipiac poll.  There has been no retraction or discussion of the question in the Jewish Journal.

                                And the bias pattern repeats with the poll of the Cato Institute.  They asked:

                                "Do you favor or oppose an agreement in which the United States and other countries that would ease oil and economic sanctions on Iran for 10-15 years in return for Iran agreeing to stop its nuclear program over that period?"

                                Their poll took place July 14-16 and they got the same result as the Qunnipiac April poll.  No correction has been issued in light of the Quinnipiac August poll results.

                                We at Avalon can assist with your statistics needs.  We use SPSS Modeler, Python and R to get the results.

                                Call us at 646 9825 9080  x1114 to get a quote.

                                Comments Disabled

                                Internet drug ratings: AskAPatient.com & WebMD correlate well but Drugs.com does not

                                Eugen G Tarnow  July 21 2015 02:42:33 PM
                                By Eugen Tarnow, Ph.D.
                                Avalon Business Systems, Inc.
                                http://AvalonAnalytics.com

                                Internet drug ratings can help consumers (ever tried reading an FDA approved drug insert?).

                                Here I compare three internet drug rating sites: AskAPatient.com, Drugs.com and WebMD.com.  For the reasons below, I recommend AskAPatient.com as well as WebMD.com

                                AskAPatient.com and WebMD.com (satisfaction rating) rates on a scale of 1-5 and Drugs.com on a scale of 1-10 so the latter ratings were scaled appropriately to enable the comparison.  The ratings for ten drugs (the "frequently rated drugs" on AskAPatient.com as of 7/22/15) are shown in Fig. 1.

                                Image:Internet drug ratings: AskAPatient.com & WebMD correlate well but Drugs.com does not
                                Fig. 1.  Ratings for 10 drugs.


                                The results are interesting.   Ratings on AskAPatient.com and WebMD.com are more negative than on Drugs.com (average rating is 2.6 and 2.8 compared to 3.8).  They are also more varied than on Drugs.com.  On AskAPatient.com the ratings vary between 1.6-3.4, on WebMD between 2.0-3.5 and on Drugs.com ratings vary between 3.4 and 4.2.  (the standard deviation for AskAPatient.com is 0.64, for WebMD it is 0.54, while for Drugs.com it is a much smaller 0.29).

                                AskAPatient.com and WebMD largely agree on their ratings (correlation is 92%) while AskAPatient.com and Drugs.com do not correlate (correlation is only 23%), see Fig. 2.

                                Image:Internet drug ratings: AskAPatient.com & WebMD correlate well but Drugs.com does not
                                Fig. 2.  Ratings for 10 drugs show little agreement between AskAPatient.com and Drugs.com


                                The ratings I compared are shown in Table 1.
                                AskAPatient.com Drugs.com WebMD.com
                                Levaquin
                                1.7
                                3.4
                                2.0
                                Lexapro
                                3.2
                                4.2
                                3.5
                                Cymbalta
                                2.8
                                3.6
                                3.0
                                Effexor
                                3.1
                                3.9
                                3.2
                                Topamax
                                3
                                3.7
                                3.3
                                Lamictal
                                3.4
                                4.2
                                3.5
                                Zyrtec
                                2.1
                                4.0
                                2.8
                                Cipro
                                1.6
                                3.9
                                2.1
                                Flagyl
                                2.5
                                3.7
                                2.6
                                Depo-Provera
                                2.1
                                3.7
                                2.4











                                Table 1.  Ratings for 10 drugs.


                                We at Avalon can assist with your statistics needs.  We use SPSS Modeler, Python and R to get the results.

                                Call us at 646 9825 9080  x1114 to get a quote.

                                  Unemployment after 2008 is hidden in a lower labor force participation rate

                                  Eugen G Tarnow  July 1 2015 02:40:39 PM
                                  By Eugen Tarnow, Ph.D.
                                  Avalon Business Systems, Inc.
                                  http://AvalonAnalytics.com

                                  We have all heard that the unemployment has gone back to levels before 2008:

                                  Image:Unemployment after 2008 is hidden in a lower labor force participation rate

                                  You may also have heard about the labor participation rate declining:

                                  Image:Unemployment after 2008 is hidden in a lower labor force participation rate

                                  Well if we combine the two, the effective unemployment rate is:

                                  Image:Unemployment after 2008 is hidden in a lower labor force participation rate

                                  As you can see, it went up after 2008 and reached a plateau in 2010.  It has not gone down since.

                                    Predictive Analytics World Chicago 2015 - savings from predictive analytics

                                    Eugen G Tarnow  July 1 2015 12:04:40 PM
                                    By Eugen Tarnow, Ph.D.
                                    Avalon Business Systems, Inc.
                                    http://AvalonAnalytics.com

                                    Predictive Analytics World in Chicago this year was a very interesting event.  The keynote speakers highlighted two ways to gain from predictive analytics.

                                    Jack Levis, Sr. Director of Process Management at UPS, showed how using predictive analytics to control the packing of the UPS trucks and the delivery paths saved them $300 million per year.  Their yearly revenue is $58 billion yielding a savings from their computer system of 0.5% and their profit is $3 billion meaning that the computer system increased the profit by 10%.  There are 500 employees working on the computer system.  It was not clear whether these salaries were included in the savings.

                                    The other keynote speaker, Rajeeve Kaul, former Senior Vice President of  OfficeMax, did a complex pricing experiment and came up with a better pricing model which saved $20-30 million that year.  Their yearly revenue was $4 billion yielding a savings from their analytics experiment of about 0.5%, just like UPS, and out of a profit of $400 million the analytics experiment was about 5%.

                                    The conclusion seems to be that if given authority over the core business, predictive analytics can save about 0.5% of the revenue and add 5-10% to the profit.

                                    Not bad.

                                    No Correlation Between Executive Pay & Shareholder Return

                                    Eugen G Tarnow  June 24 2015 10:04:13 AM
                                    By Eugen Tarnow, Ph.D.
                                    Avalon Business Systems, Inc.
                                    http://AvalonAnalytics.com

                                    Today's Wall Street Journal covers executive pay.  The journalists forgot the "line".  

                                    We generated it and it appears in the graph below of executive pay versus shareholder return.  The line is dotted and the number we were all looking for is indicated above.  R2=0.0001.

                                    In other words, there is no correlation between executive pay and shareholder return.

                                    Image:No Correlation Between Executive Pay & Shareholder Return

                                    Was the show "House, MD" correlated with vicodin use? If so, which way did the causation go?

                                    Eugen G Tarnow  May 13 2015 10:03:37 AM
                                    By Eugen Tarnow, Ph.D.
                                    Avalon Business Systems, Inc.
                                    http://AvalonAnalytics.com

                                    A very famous social psychologist, Stanley Milgram, once tried to find out whether violence on TV translates into violence in society.  His findings were that the crime rate did not go up after a crime was shown on TV, BUT that the type of crime tended to imitate the TV show.

                                    When you do predictive analytics, all kinds of ideas enter your mind when you sit and clean client data.

                                    Recently it was found that the number one drug for Medicare patients is vicodin, the drug that gets Dr. House hooked on TV.  The question I ask myself is - did the TV show influence the number of prescriptions of vicodin?  If so, in which direction?

                                    We don't have the resources to do this project, but someone ought to do it.  Help has been offered on youtube: https://www.youtube.com/watch?v=PSaasBjeJek

                                    One would need to get the daily prescription data from IMS and then the show data (from Nielsen?).  Entirely doable and fascinating project!

                                    Having a heart attack? Medicare total payments to a hospital does not guarantee a better outcome.

                                    Eugen G Tarnow  May 11 2015 09:40:58 AM
                                    By Eugen Tarnow, Ph.D.
                                    Avalon Business Systems, Inc.
                                    http://AvalonAnalytics.com

                                    Thanks to a FOIA by the Wall Street Journal, Medicare now releases datasets that describe what the tax payer gets for the yearly 0.5 trillion dollar payments.  

                                    For example, do we pay more for better quality treatments?  When it comes to heart attacks the data shows that Medicare payments hardly correlate with quality.  Below I show you the hospital total charges related to heart attack care and the corresponding "score" (% complications - lower is better).  Medicare payments represents only about 7% of the variance and even then you need a $10 million payment to lower the rate of complications from say 15% to 14%.

                                    Image:Having a heart attack?  Medicare total payments to a hospital does not guarantee a better outcome.

                                    Do hospitals that charge more per heart attack do a better job?  Here the correlation is even smaller - only 1.5% of the score is associated with price.  Medicare should probably cut the payments which range from $17,000 to $27,000 to the bottom of the range.  

                                    Image:Having a heart attack?  Medicare total payments to a hospital does not guarantee a better outcome.
                                    In other words, if you are having a heart attack, you might as well save the money in your wallet and go to a hospital that charges Medicare (and you) less for the treatment.

                                    IBM Vision - Two talks on predictive analytics

                                    Eugen G Tarnow  May 5 2015 01:44:28 PM
                                    By Eugen Tarnow, Ph.D.
                                    Avalon Business Systems, Inc.
                                    http://AvalonAnalytics.com

                                    At this month's Vision event in Orlando, there will be at least two talks about the power of SPSS Modeler.  

                                    One will be about predictive modelling at Charles Schwab: validating the "drivers" and using them for better forecasting.  There is no information yet as to what is being forecast.  The stock market?  The demand from customers?

                                    The other will be about Huffy bicycles: optimizing inventory forecasting and production schedules.

                                    Predictive analytics software used to size up patent targets, preparing a hedge fund for attack

                                    Eugen G Tarnow  April 8 2015 08:31:03 AM
                                    By Eugen Tarnow, Ph.D.
                                    Avalon Business Systems, Inc.
                                    http://AvalonAnalytics.com

                                    In today's Wall Street Journal, http://www.wsj.com/articles/hedge-fund-manager-kyle-bass-challenges-jazz-pharmaceuticals-patent-1428417408, a hedge fund is going after pharmaceutical patents and, at the same time, shorting their stock.  The idea is to use a legal channel, "Inter Partes Review", invented to invalidate patents of patent trolls, against pharmaceutical companies.

                                    How do they find the pharmaceutical companies they think are vulnerable?  Apparently "Mr. Spangenberg [consultant to the hedge fund] sizes up potential patent targets, using a predictive analytics software program he acquired several years ago."

                                    There is no description of the analytics software but here is a guess.  According to the article, the Inter Partes Review process invalidates about 77% of the patents before it.  The software, with the help of a patent lawyer, searches all those decisions for commonalities.  An analytics model is trained on those commonalities to see if it would pick out the correct 77% of invalidated patents.  Once the model is optimized it is then applied to the full patent database (which can easily be downloaded). The model then predicts which patents are most likely to be invalidated.  The hedge fund leadership then selects which companies are most vulnerable to the predicted patent failures and attacks those.

                                    A new business model in the making, created on analytics.  We can do it!

                                    The
                                    Avalon Predictive Analytics team can assist with your analytics needs.  

                                    Medicare pricing distortion raises healthcare costs by 45%

                                    Eugen G Tarnow  June 12 2014 04:15:22 PM
                                    By Eugen Tarnow, Ph.D.
                                    Avalon Business Systems, Inc.
                                    http://AvalonAnalytics.com

                                    As ObamaCare has been implemented we are finding out that even the subsidized rates are too high for many people and that the rates will keep going up.  One of the reasons for this may have nothing to do with ObamaCare but everything to do with Medicare.

                                    Medicare recently released a dataset, prompted by a freedom of information request from the Wall Street Journal, which includes most approved claims for CY2012.  My analysis of this dataset shows that Medicare causes significant price distortions.

                                    A price distribution should follow what is called a log-normal distribution (like housing prices in the UK, for example) and look like this (below).  To be as inclusive as possible, I have plotted the probability of charging a certain list price as a multiple of the Medicare largest reimbursed price – that way I can average over all Medicare procedures.
                                    Image:Medicare pricing distortion raises healthcare costs by 45%

                                    Contrast this theoretical curve with what the provider pricing actually looks like for Medicare data from 2012.  I have limited it to procedures that Medicare reimburses more than $100 per encounter:

                                    Image:Medicare pricing distortion raises healthcare costs by 45%

                                    As you can see, the left tail of the distribution is gone.  All providers, without exceptions, charge at least the Medicare largest reimbursement.  Medicare provides a price floor.  It is a customer that is so huge (14% of the population with the majority of the nation’s health problems and the time to take care of them), that a provider (91% of non-pediatric physicians accept new Medicare patients, almost all have existing Medicare patients) can always find a person covered by Medicare and would never bother with any patient who cannot pay the Medicare price.

                                    The distortion caused by the Medicare price floor increases list prices.  I can estimate just how much by fitting a log-normal distribution to the actual price distribution and I get this:

                                    Image:Medicare pricing distortion raises healthcare costs by 45%

                                    From the fit I find that if there was no Medicare price floor, the log-normal pricing curve would yield an average list price of 2.05 * Medicare.  Once Medicare comes in, all the providers raise their list prices above the Medicare price floor.  The list prices that change the most are the lowest ones.  And some providers make this particularly obvious; they simply charge multiples of Medicare (peaks in pricing at 2, and 3 times Medicare).

                                    So how much does Medicare increase the list prices?  The list price after the Medicare price floor is 2.99 * Medicare, an increase of 45% from the log-normal distribution!

                                    In NJ we have a little publicized law which states that providers can only charge 20% over Medicare for patients making less than 4 times the poverty limit.  The effect of this law is to provide yet another incentive for providers to increase their prices.

                                    What could the government do to curb ever rising health care costs (without the corresponding rise in quality)?

                                    1. Medicare rates could be set much lower to minimize distorting health care prices for everyone else.  Since current Medicare recipients only paid for 35% of its cost this would seem fair.

                                    2. Laws that say that a provider can only get the maximum reimbursement from Medicare if their list price is higher than that of Medicare have to be repealed.  This may be a form of price fixing.

                                    3.  Medicare could be thought of as a privilege and this privilege could come with a hook: if you want Medicare patients, you must also treat non-Medicare patients for the same price.

                                    The
                                    Avalon Predictive Analytics team can assist with your analytics needs.  

                                      IBM Big Data is growing at 14% a year

                                      Eugen G Tarnow  June 12 2014 04:14:05 PM
                                      By Eugen Tarnow, Ph.D.
                                      Avalon Business Systems, Inc.
                                      http://AvalonAnalytics.com

                                      I recently went to a Big Data conference hosted by IBM and things look good:  IBM's Big Data division is growing at 14% per year!

                                      The
                                      Avalon Predictive Analytics team can assist with your analytics needs.