Analytics blog from Avalon Business Systems. To find out more about our analytics offerings and SPSS Modeler please CLICK HERE.

    Chinese Mobike in Washington, D.C.: An Excellent Opportunity for Gatheting Information on Policy Makers

    Eugen G Tarnow  September 21 2017 10:29:27 AM
    By Eugen Tarnow, Ph.D.
    Avalon Business Systems, Inc.
    http://AvalonAnalytics.com


    The Chinese company Mobike is renting out bicycles in the capital.  

    The starting and end points of all bike routes are marked with the identity of the cyclist.   And the trips are smartphone activated...

    I am not familiar with Washington's goings on but I would presume this would yield a lot of intelligence on who is working with whom on various policy issues.

    Imagine what we could learn about North Korea if we rented out bicycles there?

    Comments Disabled

      Household incomes: the tail is getting longer

      Eugen G Tarnow  September 19 2017 12:07:20 PM
      By Eugen Tarnow, Ph.D.
      Avalon Business Systems, Inc.
      http://AvalonAnalytics.com

      It is frequently stated that income inequality is increasing.  Measuring this is more difficult.  Here I use one measure - the tail of the high-end household income distribution (inflation adjusted to 2017 dollars).  


      This tail is well defined for census data between $60,000-$200,000 and approximately exponential:


      Image:Household incomes: the tail is getting longer

      I can express the exponential in terms of the income difference at which the number of households decrease by half
      :

      Image:Household incomes: the tail is getting longer

      What we see is that the high end tail is getting somewhat wider.


      (There are two points for 2013 because the US Census Bureau changed the way the census was done and there are two different results.
      )
      Comments Disabled

        How come the credit rating agencies did not flag the Wells Fargo account fraud?

        Eugen G Tarnow  September 16 2017 08:49:07 AM
        By Eugen Tarnow, Ph.D.
        Avalon Business Systems, Inc.
        http://AvalonAnalytics.com


        Equifax is in the news for a colossal security breach. It seems the customers who paid for credit security were the hardest hit.


        But this may not be the biggest scandal on the horizon involving the credit rating agencies, Equifax, Experian, TransUnion, etc.


        The biggest news story might be that the credit rating agencies must have known about the Wells Fargo account fraud.  


        Imagine that suddenly one institution is creating lots of new accounts for their customers, all of which should show up in the credit rating data. Wouldn't you think credit agencies would notice it?


        There are two possibilities: (1) the credit rating agencies did know about it and chose to keep quiet or (2) the credit rating agencies are somewhat incompetent.


        Let's see which way it is.


        As usual, I reserve the right to be wrong.




        Comments Disabled

          Amazon’s Profit Picture - One Strike & It Is Out?

          Eugen G Tarnow  August 10 2017 11:37:52 AM
          By Eugen Tarnow, Ph.D.
          Avalon Business Systems, Inc.
          http://AvalonAnalytics.com

          I knew that Amazon makes most of its money from its Amazon Web Services (AWS) and not from selling products online.  But I never really knew the numbers.  Here they are:

          Image:Amazon’s Profit Picture - One Strike & It Is Out?

          Amazon makes the bulk of its revenue from non-AWS sources but the most of the profit comes from AWS and the profit margin is all AWS.  In fact, the profit margin on non-AWS revenue is only 0.9%, about the same as for a supermarket.

          The acquisition of Whole Foods with a profit margin of about 3% then makes Amazon less vulnerable to a downturn. But is seems that if there is ever a long strike at Amazon, it might just kill the business. As a comparison, the profit margin of Alphabet is 16%, though declining 4% per year.

          But Amazon also may be an example of why worker salaries are increasing so slowly: There just is no room for salary increases at Amazon and since they are killing the competition, neither is there room for salary increases elsewhere in retail.

          The shareholders are presumably hoping that Amazon will kill its competition and then raise the prices and make a real profit. But if it raises the prices on the customers it may open the door to anti-trust action...

          Numbers are taken from https://revenuesandprofits.com/amazon-revenues-profits-analysis-2017-update/

          As always I reserve the right to be wrong.
          Comments Disabled

            Amazon’s Reverse Machine Learning Algorithm Annoys Customer

            Eugen G Tarnow  August 10 2017 09:27:24 AM
            By Eugen Tarnow, Ph.D.
            Avalon Business Systems, Inc.
            http://AvalonAnalytics.com

            Machine Learning is still in its infancy at Amazon - at least when it comes to customer support.  

            I had ordered an Amazon certified lightning cord for my iPhone and never received it.  There was no way for me to communicate this issue - the only complaint one could file was to return the item.  Since I never received the item I could not return it.  

            This already raised my eyebrows: The quality of the Amazon customer support software is certainly below par if it does not allow the user to indicate that an item was never received.

            I decided to put up a review (below) and see if that would work.

            The Amazon machine learning algorithm rejected my review and foisted the responsibility of figuring out why it was rejected (I term this "Reverse Machine Learning") onto me, the customer.   Presumably it had to do with the ALL CAPS subject line.  But then it should have just said so.  And that rule would be too simple since some old people use just CAPS.

            Not a very attractive use of machine learning and reversing it on the customer - not a good idea.

            Amazon machine learning email:
            Amazon style=


             


             


            Your review could not be posted.

            Thanks for submitting a customer review on Amazon. Your review could not be posted to the website in its current form. While we appreciate your time and comments, reviews must adhere to the following guidelines:
            http://www.amazon.com/review-guidelines


            AmazonBasics Apple Certified Lightning to USB Cable - 3 Feet (0.9 Meters), Black ★  from Something on August 9, 2017

            NEVER RECEIVES SOMETHING AND THERE IS NO WAY TO COMPLAIN


            I never received it and there is no way for me to communicate with Amazon to tell them


             


             


            We encourage you to revise your review and submit it again. A few common issues to keep in mind:
            • Your review should focus on specific features of the product and your experience with it. Feedback on the seller or your shipment experience should be provided at www.amazon.com/feedback.
            • We do not allow profane or obscene content. This applies to adult products too.
            • Advertisements, promotional material or repeated posts that make the same point excessively are considered spam.
            • Please do not include URLs external to Amazon or personally identifiable content in your review.
             


            This message is for: etarnow@avabiz.com. If you'd rather not receive future e-mails of this sort from Amazon.com, please opt-out here.
            Reference ATVPDKIKX0DER-R21PPOAHIVZ9MG. (c) 2014 Amazon.com, Inc. or its affiliates. Amazon.com, 410 Terry Avenue N., Seattle, WA 98109-5210.



            Comments Disabled

              Correlation between Medicaid Expansion And Overdose Deaths

              Eugen G Tarnow  July 18 2017 08:01:05 PM
              By Eugen Tarnow, Ph.D.
              Avalon Business Systems, Inc.
              http://AvalonAnalytics.com

              Some states chose to expand Medicaid under ObamaCare and some did not.  Those who chose to expand Medicaid tend to have more deaths from overdoses (28% more, p<0.03).


              Image:Untitled

              It is unclear why this relationship holds.
              Comments Disabled

                Google Does Not Scale

                Eugen G Tarnow  June 28 2017 07:15:54 AM
                By Eugen Tarnow, Ph.D.
                Avalon Business Systems, Inc.
                http://AvalonAnalytics.com

                In the aftermath of the EU decision against Google's manipulation of its search results to favor its own products, it is instructive to ask why Google did what it did.

                The answer is, perhaps surprising: Google had to do it because Google does not scale. There are too many web pages and Google can't figure out what the order of the search results should be, including where its own products should display.

                Let me repeat it.  The Google search algorithm does not scale.  

                If it did scale, Google would not have to change its algorithm all the time.

                The reason it does not scale is that Google does not have enough statistics of what search results we prefer to click on.  The reason for that is that we rarely go beyond the first 5-10 results, making results further down the list for all purposes nonsense to Google.  Google just has no idea whether search result 15 should be higher up than result 16 because neither get enough clicks.  And Google has really no idea whatsoever once we get to search result 301.  

                Thus Google functions sort of as an intelligent phone book with 10 results that may be relevant.  The rest might as well never show up.

                If Google wants to sell its own products and the relevant search result starts out as 301, it will never move up.  So Google has to cheat and move it up to bypass its own search algorithm which does not scale.

                Here is an example.  I wanted to know who purports to measure the time it takes to fill out government forms.  My search query was "who measures how long it takes to fill out a government form".  The 2nd result was "How the Government Measures Unemployment".  The 8th result was "Handbook of Sexuality-Related Measures".  I did not find the answer I was looking for.

                So Google is great for answering some popular questions, to find the five largest manufacturers of a particular gadget. and to direct users to the businesses in Google's business directory (as long as they get to push their own directory ahead of everything else).

                But if you are a small business trying to sell something you don't have a chance.  And don't let the SEO companies tell you otherwise.  

                Google makes everybody scurry around in the false belief they can show up on the first search page.  It changes its algorithm and makes everybody run around again.

                It is just a waste of time - because Google does not scale.

                Added September 1, 2017: Google does not seem to know how to alphabetize.  A search on Big Little Lies indicates that the first choice of outlet starts with a "Y":
                Image:Google Does Not Scale.

                Comments Disabled

                  A new algorithm for promotion optimization?

                  Eugen G Tarnow  June 26 2017 06:31:52 PM
                  By Eugen Tarnow, Ph.D.
                  Avalon Business Systems, Inc.
                  http://AvalonAnalytics.com

                  We seem to have discovered a new algorithm for promotion optimization.  

                  Algorithms are not patentable so we keep them proprietary.  

                  But if anyone is interested in comparing their algorithm to ours, we will be happy to participate.  



                  Comments Disabled

                    Getting 3% growth by increasing the workforce: have fewer people go to college and abbreviate high school

                    Eugen G Tarnow  June 25 2017 09:07:15 AM
                    By Eugen Tarnow, Ph.D.
                    Avalon Business Systems, Inc.
                    http://AvalonAnalytics.com

                    Rex Nutting of MarketWatch wrote an opinion piece titled "Sorry, Mr. Trump, but the only way to get to 3% growth is to hire more Mexicans ".

                    He stated that in order to get that 3% growth we would have to hire another 33 million workers of the next ten years.

                    We can get there half-way by having companies stop insisting on college credentials that are not needed in the workplace.

                    Today 20 million people are attending college.  If we cease being a "credentialist" society, and, over ten years, can get say 75% of college students working instead of studying, then we will get 15 million workers into the workplace.

                    And think of how many fewer people would be burdened by those student loans...

                    Another 25% of what is needed can be had by abbreviating high school (there are 15 million in high school at any one time). Much of what is studied there is not needed in the workplace either.  Cut two years off of high school and another 4 million workers can be added.

                    As always, I reserve the right to be wrong...




                    Comments Disabled

                      Earnings correlate less and less with cognitive ability - at least for Finnish men - or is it just Nokia?

                      Eugen G Tarnow  June 25 2017 06:59:47 AM
                      By Eugen Tarnow, Ph.D.
                      Avalon Business Systems, Inc.
                      http://AvalonAnalytics.com

                      Evidence just arrived showing that earnings at 30 are correlating less and less with cognitive ability. At least among Finish men since the year 2000.  (See Jokela, M., Pekkarinen, T., Sarvimäki, M., Terviö, M., & Uusitalo, R. (2017). Secular rise in economically valuable personality traits. Proceedings of the National Academy of Sciences, 201609994.)

                      Finnish men were tested for their personality traits and their cognitive abilities as they were ready to join the armed forces draft.  Below is displayed the correlations within each cohort of the test results with earnings at 30.  

                      Image:Earnings correlate less and less with cognitive ability - at least for Finnish men - or is it just Nokia?

                      So right around the tech bubble, the correlation of cognitive ability with pay peaked.

                      But wait.  Finland is known to be a one-company country (see http://www.economist.com/node/21560867 ).  This is what happened to Nokia stock around the tech bubble:

                      Image:Earnings correlate less and less with cognitive ability - at least for Finnish men - or is it just Nokia?

                      Could it be that the earnings findings just pick up what is going on at Nokia?  


                      Comments Disabled

                        WSJ’s Daily Shot covered our research

                        Eugen G Tarnow  June 22 2017 09:49:37 AM
                        By Eugen Tarnow, Ph.D.
                        Avalon Business Systems, Inc.
                        http://AvalonAnalytics.com

                        The Wall Street Journal's Daily Shot covered our research.  See https://blogs.wsj.com/dailyshot/2017/06/22/wsjs-daily-shot-us-shale-oil-output-to-hit-record-high-in-july/ and scroll to "Avalon".


                        Comments Disabled

                          Colleges improve critical thinking only up to the 1250 score on the CLA+

                          Eugen G Tarnow  June 5 2017 04:17:45 PM
                          By Eugen Tarnow, Ph.D.
                          Avalon Business Systems, Inc.
                          http://AvalonAnalytics.com

                          The Wall Street Journal published the CLA+ scores of 68 colleges. CLA+ is a test to measure the ability to think critically.  

                          I graphed the score improvements as a function of the incoming class score.  It appears that colleges with low scoring incoming classes are able to increase their score but this increase disappears as the students get smarter.  

                          At 1250 colleges no longer improve the score averages.

                          Image:Colleges improve critical thinking only up to the 1250 score on the CLA+
                          Comments Disabled

                            Facebook - a drag on productivity?

                            Eugen G Tarnow  June 2 2017 09:42:24 PM
                            By Eugen Tarnow, Ph.D.
                            Avalon Business Systems, Inc.
                            http://AvalonAnalytics.com

                            The New York Times wrote that "Facebook Has 50 Minutes of Your Time Each Day. It Wants More.".  

                            That translates into 100 hours a quarter.  

                            Facebook is able to make $20 per US user for those 100 hours.  

                            That translates into $0.20 per hour.  

                            Suppose the advertisers spend 10% of their revenue marketing.  If Facebook ads are of average utility then the advertisers make $2 per hour.  One hour of your Facebook time is then creating $2.20 value for others.

                            Thus an hour worth $20 is transformed into an hour worth just $2.20:

                            Image:Facebook - a drag on productivity?

                            Data scientists and data engineers working for Facebook attempt to keep your attention within Facebook.  

                            That also means they are trying to remove 89% of the value you could derive from your time.

                            The Zuckerberg proposal of a "Universal Basic Income" seems to be in line with this - the UBI would remove much of the competition for attention allowing Facebook to get more user time.

                            What did we forget?  Can the consumer surplus save the equation?  If would seem that we would need a very large consumer surplus.

                            It seems that this is a relatively new issue.  Video games and Netflix binge watching seems to fall in the same category - both are designed to keep the user from leaving and probably do not make much money per hour either.

                            As always, I reserve the right to be wrong.




                             
                            Comments Disabled

                              CEO pay for S & P 500 firms: Women paid same as men & have the same one-year return. Still no correlation between pay and share holder value.

                              Eugen G Tarnow  May 31 2017 10:32:59 AM
                              By Eugen Tarnow, Ph.D.
                              Avalon Business Systems, Inc.
                              http://AvalonAnalytics.com

                              The Wall Street Journal just published the CEO pay and share holder return for the S & P 500 companies.  Their conclusion (women CEOs are paid more) is different from mine: the women are not paid more than the men - they are paid the same, at least if one adjusts for statistical noise.  

                              Neither gender is better for share holders.

                              Here is the result of my ANOVA:
                              Field Men Women p (useful if <0.05)
                              Total CEO Pay ($million)
                              13.288
                              14.346
                              0.65
                              ShareHolderReturn (%)
                              0.143
                              0.232
                              0.10




                              And here are the graphs:
                              Image:CEO pay for S & P 500 firms: Women paid same as men & have the same one-year return.  Still no correlation between pay and share holder value.Image:CEO pay for S & P 500 firms: Women paid same as men & have the same one-year return.  Still no correlation between pay and share holder value.

                              Image:CEO pay for S & P 500 firms: Women paid same as men & have the same one-year return.  Still no correlation between pay and share holder value.Image:CEO pay for S & P 500 firms: Women paid same as men & have the same one-year return.  Still no correlation between pay and share holder value.

                              As usual, there is no correlation between CEO pay and share holder return:

                              Image:CEO pay for S & P 500 firms: Women paid same as men & have the same one-year return.  Still no correlation between pay and share holder value.






                              Comments Disabled

                                The Best New Jersey Soccer Clubs

                                Eugen G Tarnow  May 18 2017 09:22:14 AM
                                By Eugen Tarnow, Ph.D.
                                Avalon Business Systems, Inc.
                                http://AvalonAnalytics.com

                                Which are the best kids' soccer clubs in New Jersey?  Some clubs have a great 2003 team and a terrible 2002 team - can you even compare them?

                                Perhaps it is just chance?

                                It turns out it is not chance.  I compared the clubs that had the ten highest nationally ranked 2003 teams out of which I kept those with at least four nationally ranked teams in different age groups.  I performed ANOVAs on the inverse ranking and on the log(ranking) and found that, indeed, there is a statistically significant difference in both cases (p<0.0008).

                                The club rankings come out slightly differently depending upon how you do the averaging.  On top is displayed the inverse of the inverse average and on bottom is displayed the exponential of the log averages.  

                                The winner is Match Fit, second place is taken by TSF and third place is shared by FC Copa and Morris United..


                                Image:The Best New Jersey Soccer Clubs
                                Image:The Best New Jersey Soccer Clubs
                                Comments Disabled

                                  When calling persistence pays off

                                  Eugen G Tarnow  May 9 2017 10:30:52 AM
                                  By Eugen Tarnow, Ph.D.
                                  Avalon Business Systems, Inc.
                                  http://AvalonAnalytics.com

                                  When making sales calls persistence pays off.  Here is what happens when bank sales people call potential customers to get a term deposit.

                                  Image:When calling persistence pays off
                                  Comments Disabled

                                    Attractiveness or a barrier?

                                    Eugen G Tarnow  May 3 2017 12:17:52 PM
                                    By Eugen Tarnow, Ph.D.
                                    Avalon Business Systems, Inc.
                                    http://AvalonAnalytics.com

                                    Louis Vuitton's VP of Analytics, Carey Chou, performed a fascinating and thorough analysis of their web sales data and found that some products act as attractors to other products.  See this example (taken from https://sharedmemory.github.io/blog/2016/10/13/customer-choice-model-and-attraction-effect/ ):


                                    Image:Attractiveness or a barrier?
                                    The probability that the customer will buy a particular bag is changed as a third alternative is introduced.  This third alternative makes the first bag more attractive and the second bag less attractive.  Note that the probability of sale is extremely low for this third bag.

                                    But perhaps the third bag acts somewhat as a barrier preventing the user from scanning to the right?  The effects are small but the ideas are interesting.
                                    Comments Disabled

                                      Facebook, MIT, Gender & Rejection

                                      Eugen G Tarnow  May 3 2017 05:36:07 AM
                                      By Eugen Tarnow, Ph.D.
                                      Avalon Business Systems, Inc.
                                      http://AvalonAnalytics.com


                                      Image:Facebook, MIT, Gender & Rejection
                                      Source:
                                      https://www.wsj.com/articles/facebooks-female-engineers-claim-gender-bias-1493737116
                                      http://www.collegeconfidential.com/dean/000255/
                                      Comments Disabled

                                        GNP seasonality adjustments appear to be just fine

                                        Eugen G Tarnow  April 30 2017 09:34:17 AM
                                        By Eugen Tarnow, Ph.D.
                                        Avalon Business Systems, Inc.
                                        http://AvalonAnalytics.com

                                        The Wall Street Journal questioned the GDP seasonal adjustment methods as the latest quarter came in at a low 0.7% growth and it seemed like the first quarter often comes in low:

                                        Image:GNP seasonality adjustments appear to be just fine

                                        An ANOVA test of their hypothesis, however, shows that there is no significant difference between the quarters (p=0.18) so the seasonal adjustment appears to work just fine.  

                                        (I read the numbers off of the graph as:
                                        1st quarter 2nd quarter 3rd quarter 4th quarter Year
                                        3.75
                                        2.75
                                        2.45
                                        2010
                                        -1.5
                                        3.25
                                        0.75
                                        4.5
                                        2011
                                        2.75
                                        1.875
                                        0.5
                                        0.1
                                        2012
                                        2.8
                                        0.8
                                        3.1
                                        4
                                        2013
                                        -1
                                        4
                                        5
                                        2.2
                                        2014
                                        2
                                        2.65
                                        2
                                        0.85
                                        2015
                                        0.8
                                        1.4
                                        3.5
                                        2.1
                                        2016
                                        0.7
                                        2017






                                        ).
                                        Comments Disabled

                                          "Purdue Pharma will partner with the Commonwealth of Virginia to enhance its prescription monitoring program to address opioid abuse."

                                          Eugen G Tarnow  April 18 2017 03:58:01 PM
                                          By Eugen Tarnow, Ph.D.
                                          Avalon Business Systems, Inc.
                                          http://AvalonAnalytics.com

                                          Purdue Pharma is one of the largest opioid distributors.  

                                          I guess that means Purdue Pharma has the Commonwealth of Virginia in its pocket.  See http://healthitanalytics.com/news/prescription-monitoring-program-to-reduce-opioid-abuse-in-va
                                          Comments Disabled

                                            A Perfect Storm - US Health Care

                                            Eugen G Tarnow  April 10 2017 12:02:11 PM
                                            By Eugen Tarnow, Ph.D.
                                            Avalon Business Systems, Inc.
                                            http://AvalonAnalytics.com

                                            Vanguard just sent out a newsletter that identifies "the structural forces dampening global growth and interest rates".  

                                            They are:
                                            • Technology - falling technology costs
                                            • Demographics - an aging populace
                                            • Globalization - competition on prices and labor

                                            Thus far health care has been immune to, or benefited from, all three forces.

                                            Health care technology costs are typically increasing, not decreasing, and used not to eliminate inefficiencies but to open up new product lines.  Since the 1970s a substantial number of doctors could have been replaced by computers since AI systems were already then giving good enough diagnoses.  AI diagnostic systems have improved but their use is always limited by some kind of licensing requirement. At the same time, electronic medical records have become more and more expensive and cumbersome.  They do not include a diagnostic field presumably to prevent AI from taking over.

                                            The demographics will increase the need for medical care. After Medicare was introduced, doctor salaries increased from three times the average salary to seven times ( http://www.medpagetoday.com/Columns/At-Large/27158 ).  The demographic change will, no doubt, bring this inequality to new heights.

                                            Health care has been able to escape globalization by a variety of protective laws.  Drugs cannot be imported, FDA prevents European approved drugs from being used, doctors are very hard to import (unlike computer programmers) and that is restricted also by state lines...

                                            That adds up to the perfect storm.  While most families in the US have had their salary growth stop due to technology and globalization, health care has not and demographic changes will exasperate this.

                                            Health care will give us the perfect storm.

                                            Image:A Perfect Storm - US Health Care
                                            Figure from http://www.mnhospitals.org/Portals/0/Documents/education/7016_GS01.pdf
                                            Comments Disabled

                                              Is there something fishy with Johns Hopkins University’s Pima Indians Diabetes Data Set?

                                              Eugen G Tarnow  April 7 2017 11:43:47 AM
                                              By Eugen Tarnow, Ph.D.
                                              Avalon Business Systems, Inc.
                                              http://AvalonAnalytics.com

                                              This is a famous data set describing the incidence of diabetes in a population prone to diabetes.  It can be downloaded from here: https://archive.ics.uci.edu/ml/datasets/Pima+Indians+Diabetes .  

                                              Some thirty data science publication have resulted.  But there are some strange things going on in this dataset.  

                                              First, the age distribution of the participants is exponential:

                                              Image:Is there something fishy with Johns Hopkins University’s Pima Indians Diabetes Data Set?

                                              Second, the body mass index does not increase with age:

                                              Image:Is there something fishy with Johns Hopkins University’s Pima Indians Diabetes Data Set?

                                              I wrote the dataset depositor but did not receive an answer.  I wrote the archivists at University of California Irvine and they decided to just leave the dataset up.

                                              But it seems there is something very wrong with it.  An the publications that resulted - are they therefore wrong too?

                                              As always, I reserve the right to be wrong.

                                              Comments Disabled

                                                Is Restless Leg Syndrom based on a flawed survey?

                                                Eugen G Tarnow  April 6 2017 02:25:21 PM
                                                By Eugen Tarnow, Ph.D.
                                                Avalon Business Systems, Inc.
                                                http://AvalonAnalytics.com

                                                When one constructs a survey it is important that the questions are not leading to the answer.  

                                                This is not the case with the "International Restless Leg Syndrome Rating Scale" - see http://www.rls.org.au/pdf/PKGD6.pdf ).

                                                Each question suggests to the patient that they have Restless Leg Syndrome (RLS).  For example:

                                                "1. Overall, how would you rate the RLS discomfort in you legs or arms?"


                                                "2. Overall, how would you rate the need to move around because of your RLS symptoms?"


                                                etc.

                                                This survey was produced by "The International Restless Legs Syndrome Study Group" and "validated" in a journal called SleepMedicine (see http://www.sleep-journal.com/article/S1389-9457(02)00258-7/abstract ).

                                                My guess is that there is a substantial overlap between high scores on the International Restless Leg Syndrome Rating Scale and and suggestibility.

                                                No wonder there are a lot of people who perceive themselves to suffer from RLS.

                                                No wonder there is no known cause of RLS.

                                                And no wonder that the placebo effect is so large: a-clinical-trial-with-humorous-results

                                                But seriously, how much is this costing us tax payers?
                                                Comments Disabled

                                                  A clinical trial with humorous results

                                                  Eugen G Tarnow  March 31 2017 03:43:44 PM
                                                  By Eugen Tarnow, Ph.D.
                                                  Avalon Business Systems, Inc.
                                                  http://AvalonAnalytics.com

                                                  We humans exist on a normal distribution and it is the job of healthcare marketers to keep chipping away at the tails.  

                                                  One such effort is the "Restless Leg Syndrome".  So GlaxoSmithKline, the National Institute of Health, the FDA and somebody else whose logo I cannot identify

                                                  Image:A clinical trial with humorous results

                                                  set out to perform a clinical trial.  They used the "International Restless Legs Syndrome Rating Scale":

                                                  http://www.rls.org.au/pdf/PKGD6.pdf

                                                  The maximum score is 40.  It does not say in the clinical trial results what the baseline scores were.

                                                  https://www.clinicaltrialsregister.eu/ctr-search/rest/download/result/attachment/2005-005372-32/1/2034

                                                  The results were overwhelming:

                                                  Image:A clinical trial with humorous results

                                                  As soon as the treatment period started, RLS patients started to improve!  The improvement kept increasing until about 8 weeks into the trial where it stopped at an impressive 12 points with the placebo and 15 points with the drug!

                                                  The upshot is then that anybody with RLS should ask a friend to start a "clinical trial" on the sufferer - and her/his leg will stay put.

                                                  Comments Disabled

                                                    Is the opioid epidemic a rural issue?

                                                    Eugen G Tarnow  March 29 2017 01:10:33 PM
                                                    By Eugen Tarnow, Ph.D.
                                                    Avalon Business Systems, Inc.
                                                    http://AvalonAnalytics.com

                                                    It was suggested to me that the opioid epidemic may concentrated in the rural areas of the US.  Interestingly, there is no correlation between Medicare opioid claims in a state and how rural the population is:

                                                    Image:Is the opioid epidemic a rural issue?

                                                    There is also no correlation between the rate of overdose deaths and how rural the population is:

                                                    Image:Is the opioid epidemic a rural issue?
                                                    Comments Disabled

                                                      A Press Release on Opioids - Record Number of New Jersey Doctors Disciplined in 2016 As State Ramps up Efforts to Curb Opioid Addiction

                                                      Eugen G Tarnow  March 21 2017 10:28:23 AM
                                                      By Eugen Tarnow, Ph.D.
                                                      Avalon Business Systems, Inc.
                                                      http://AvalonAnalytics.com

                                                      The New Jersey Department of the Attorney General issued a press release earlier this month about 31 medical doctors disciplined for opioid issues (you have to ask for the actual 31 names, they are not in the press release but you can find them on the bottom of this blog). It states "Purging the medical community of over-prescribers is as important to our cause as busting heroin rings and locking up drug kingpins.”

                                                      Well, if that is the case, few kingpins would be locked up and then for only a short time.

                                                      All but 5 of the doctors had prescribed opioids within Medicare.  The average number of Medicare prescriptions for these doctors were 650 per year.  327 NJ doctors had prescribed more than that within Medicare which suggests a conviction rate of 10%.

                                                      The majority of opioid sanctions were temporary: 61%.

                                                      The doctors sanctioned for opioids have somewhat more sanctions than the average sanctioned doctor (1.9 versus 1.5).  One doctor had 8 sanctions altogether - 7 previous sanctions.

                                                      That the state is indeed ramping up its efforts is shown by a graph of the number of doctors sanctioned by month covered by the press release - it is increasing:
                                                      Image:A Press Release on Opioids - Record Number of New Jersey Doctors Disciplined in 2016 As State Ramps up Efforts to Curb Opioid Addiction

                                                      However, the total number of sanctions by the medical board is the same over the last ten years, about 114 (which probably works out to about 0.4% of all active physicians in NJ):
                                                      Image:A Press Release on Opioids - Record Number of New Jersey Doctors Disciplined in 2016 As State Ramps up Efforts to Curb Opioid Addiction

                                                      There were also personal tragedies among the doctors - at least one had a child who overdosed on the doctor's opioid pills.

                                                      Here is an interesting question: will anybody sue these 31 doctors and be able to collect from their malpractice insurance?  Will the insurance companies then represent these doctors?

                                                      Many of the doctors had multiple versions of their names, one even had the first and last names switched in the different data sets.  Furthermore, not all sanctions could be found on the website of the State Board of the Medical Examiners (http://www.njconsumeraffairs.gov/bme/Pages/actions.aspx). One wonders whether data quality issues make it harder to catch perpetrators.

                                                      Final 2016 CDS Case List.docx
                                                      Comments Disabled

                                                        The Art of the Deal: Sweden pays the least for drugs in all of Europe (and much much less than the US)

                                                        Eugen G Tarnow  February 24 2017 03:06:02 PM
                                                        By Eugen Tarnow, Ph.D.
                                                        Avalon Business Systems, Inc.
                                                        http://AvalonAnalytics.com

                                                        When looking at Sweden, there are many gems to be found.  One is how they negotiate their drug prices.  It is something that we all should take a look at.

                                                        An Epipen is $40 and nobody forces you to buy 2 of them.  In the US a pair is $700 and you have to buy both.

                                                        See: http://www.tlv.se/Upload/Ovrigt/Rapport_internationell_prisjamforelse_lakemedel_151215.pdf


                                                        Image:The Art of the Deal: Sweden pays the least for drugs in all of Europe (and much much less than the US)

                                                        For prices of particular drugs, see: http://www.tlv.se/beslut/sok/lakemedel/

                                                        Call your politicians and your journalist friends and ask them to cover this.


                                                        Comments Disabled

                                                          Algebra 1 Scores & Social Group: Differences Increase from 2015 to 2016

                                                          Eugen G Tarnow  February 21 2017 01:43:24 AM
                                                          By Eugen Tarnow, Ph.D.
                                                          Avalon Business Systems, Inc.
                                                          http://AvalonAnalytics.com

                                                          As income inequality increases, one would expect standardized test scores to do the same.  In the NJ PARCC Alegra 1 test, this is indeed the case.  Here are the 2015 scores by social group:

                                                          Image:Algebra 1 Scores & Social Group: Differences Increase from 2015 to 2016

                                                          And here are the differences between 2016 and 2015

                                                          Image:Algebra 1 Scores & Social Group: Differences Increase from 2015 to 2016

                                                          As the social group increases from A to J, the scores go up.  This corresponds to 73% of the variation:

                                                          Image:Algebra 1 Scores & Social Group: Differences Increase from 2015 to 2016

                                                          As always, I reserve the right to be wrong.
                                                          Comments Disabled

                                                            H1B Salary Distributions

                                                            Eugen Tarnow  February 2 2017 11:45:46 AM
                                                            By Eugen Tarnow, Ph.D.
                                                            Avalon Business Systems, Inc.
                                                            http://AvalonAnalytics.com

                                                            The salary distributions of H1B workers - for single applications looks like this.  The most common salary is $60-65,000.
                                                            Image:H1B Salary Distributions
                                                            For applications with two workers needed at the same time the distribution narrows and, again, the most common salary is $60-65,000 per year.
                                                            Image:H1B Salary Distributions
                                                            In Silicon Valley the distribution for single applications is centered around $100,000:

                                                            Image:H1B Salary Distributions
                                                            For applications with two workers it is much lower with the most common salary between $75-80,000:
                                                            Image:H1B Salary Distributions

                                                            In New York City, the single application distribution is as follows:

                                                            Image:H1B Salary Distributions

                                                            and the two-worker application distribution is:

                                                            Image:H1B Salary Distributions

                                                            The most common salary in the second case is $70-75,000.  

                                                            Cognizant typically applies for 30 workers at a time.  For those applications the salary distributions look like:

                                                            Image:H1B Salary Distributions
                                                            The most common salary is $60-65,000.

                                                            The data was taken from https://www.foreignlaborcert.doleta.gov/performancedata.cfm using fiscal year 2015 and when the employer gives a range in pay, the lowest number was chosen.
                                                            Comments Disabled

                                                              Which Occupations Need The Cheapest H1B Salaried Workers?

                                                              Eugen G Tarnow  January 31 2017 01:56:59 PM
                                                              By Eugen Tarnow, Ph.D.
                                                              Avalon Business Systems, Inc.
                                                              http://AvalonAnalytics.com

                                                              One might be curious which occupations the US just don't have enough workers in but at the same time are so easy to find abroad that they can be paid the minimum allowable H1B salary? The majority is programmers and analysts - no need to focus on the physical therapists (???).

                                                              Image:Which Occupations Need The Cheapest H1B Salaried Workers?

                                                              The data was taken from https://www.foreignlaborcert.doleta.gov/performancedata.cfm using fiscal year 2015 and when the employer gives a range in pay, the lowest number was chosen.
                                                              Comments Disabled

                                                                Which US Counties Have No Smart STEM People & Need The Most H1B Workers At Minimum Allowable Salary?

                                                                Eugen Tarnow  January 31 2017 01:44:13 PM
                                                                By Eugen Tarnow, Ph.D.
                                                                Avalon Business Systems, Inc.
                                                                http://AvalonAnalytics.com

                                                                Which counties just don't have enough smart workers so they have to hire minimum salary H1B workers?  Here is a list and it includes places with the lowest STEM ability like Silicon Valley, New York, King (where Microsoft is), Dallas, San Francisco, Bergen (where I live).  

                                                                Image:Which US Counties Have No Smart STEM People & Need The Most H1B Workers At Minimum Allowable Salary?

                                                                The data was taken from https://www.foreignlaborcert.doleta.gov/performancedata.cfm using fiscal year 2015 and when the employer gives a range in pay, the lowest number was chosen.
                                                                Comments Disabled

                                                                  Which Attorneys File The Most Applications Paying H1B Workers the Legal Minimum?

                                                                  Eugen G Tarnow  January 31 2017 01:30:16 PM
                                                                  By Eugen Tarnow, Ph.D.
                                                                  Avalon Business Systems, Inc.
                                                                  http://AvalonAnalytics.com

                                                                  One might be curious who are the lawyers that file the most H1B applications with the minimum legal limit?  See below.

                                                                  The data was taken from https://www.foreignlaborcert.doleta.gov/performancedata.cfm using fiscal year 2015 and when the employer gives a range in pay, the lowest number was chosen.

                                                                  Image:Which Attorneys File The Most Applications Paying H1B Workers the Legal Minimum?



                                                                  Comments Disabled

                                                                    Which Companies Pay Less Than 1% Over Legal Minimum for H1B Workers?

                                                                    Eugen Tarnow  January 31 2017 01:11:46 PM
                                                                    By Eugen Tarnow, Ph.D.
                                                                    Avalon Business Systems, Inc.
                                                                    http://AvalonAnalytics.com

                                                                    Which companies pay their H1B workers the very minimum?  Here is a list.  The largest number of minimum wage applications come from Indian companies.  But we have American ones as well:  PWC, Deloitte, Accenture, CapGemini, Apple, Oracle, Cisco.

                                                                    Image:Which Companies Pay Less Than 1% Over Legal Minimum for H1B Workers?

                                                                    Do you think our colleges are any different?  Here is a list of the colleges with the most workers with minimum pay:

                                                                    Image:Which Companies Pay Less Than 1% Over Legal Minimum for H1B Workers?

                                                                    The data was taken from https://www.foreignlaborcert.doleta.gov/performancedata.cfm using fiscal year 2015 and when the employer gives a range in pay, the lowest number was chosen.
                                                                    Comments Disabled

                                                                      Statistics of the H1B Program: Most H1B Workers Get Minimum Pay

                                                                      Eugen G Tarnow  January 31 2017 12:18:36 PM
                                                                      By Eugen Tarnow, Ph.D.
                                                                      Avalon Business Systems, Inc.
                                                                      http://AvalonAnalytics.com

                                                                      The H1B program is a goldmine for the corporations participating.  While it is supposed to supplant the lack of STEM talent in the US, the evidence is not there.  

                                                                      Here is the promised salaries for single and multiple H1B applications.  Note that 44% (72%) of single (multiple) applications pay less than 1% over the prevailing wage, which is the legal minimum pay.

                                                                      The data was taken from https://www.foreignlaborcert.doleta.gov/performancedata.cfm using fiscal year 2015.

                                                                      Image:Statistics of the H1B Program: Most H1B Workers Get Minimum Pay


                                                                      Comments Disabled

                                                                        School District Income Seems to Mean Little to the Poor

                                                                        Eugen G Tarnow  January 25 2017 06:50:15 PM
                                                                        By Eugen Tarnow, Ph.D.
                                                                        Avalon Business Systems, Inc.
                                                                        http://AvalonAnalytics.com

                                                                        I had a conversation with a school board member in a New Jersey district in which the mean family income is large but economically disadvantaged students do not do well.  I decided to see if this was a larger phenomenon.

                                                                        Indeed it was.  For example, below is shown the grade 6 PARCC math scores as a function of district family income for all children (top) and for economically disadvantaged children (bottom).  There is a strong correlation between the math scores and the district family income in general (R square is 44%) but not for the poor (R square is only 2%).  

                                                                        It is also interesting just how expensive it is to increase the average grade by 1 point: $5,000 in mean income!

                                                                        Image:School District Income Seems to Mean Little to the Poor
                                                                        Comments Disabled

                                                                          Student Loans Go Missing

                                                                          Eugen G Tarnow  January 19 2017 12:59:59 PM
                                                                          By Eugen Tarnow, Ph.D.
                                                                          Avalon Business Systems, Inc.
                                                                          http://AvalonAnalytics.com

                                                                          Student loans are in the news because the Department of Education had a programming glitch that understated the number of delinquent student loans.

                                                                          It is interesting to see how the student loans become delinquent.  I used the data from https://studentaid.ed.gov/sa/about/data-center/student/portfolio .

                                                                          Student loans that are 31-90 days delinquent have a 56% chance of becoming delinquent 91-180 days, then a 57% chance of becoming delinquent 181-270 days and then a 70% chance of becoming delinquent 271-360 days.  

                                                                          After that the loans are to be transferred into collection.  The number of loans in collection is not reported, only the number of loans transferred into collection.

                                                                          This is where it becomes interesting.  One would think that the loans would go into collection as soon as they reach 361 days.  But after a quarter, there is no correlation between the delinquencies and the collections (all in billions):

                                                                          Image:Student Loans Go Missing

                                                                          Rather, if we plot the linear coefficient between delinquencies and collections as a function of the time delay, we find that it takes 6 quarters before there is a relationship between the two and then the relationship is a negative one, the more delinquencies, the fewer of them go to collection:

                                                                          Image:Student Loans Go Missing

                                                                          If we plot the missing loans as a function of quarter we find that it keeps rising:

                                                                          Image:Student Loans Go Missing

                                                                          If we add up the total missing loans after so many quarters we get:

                                                                          Image:Student Loans Go Missing

                                                                          Forecasting by extrapolation we get:

                                                                          Image:Student Loans Go Missing

                                                                          It will reach $1 Trillion in 68 quarters from now, in 17 years.  

                                                                          As always, I reserve the right to be wrong.
                                                                          Comments Disabled

                                                                            NJ PARCC Algebra 1 Scores Changes Over A Year

                                                                            Eugen G Tarnow  January 4 2017 06:49:41 PM
                                                                            By Eugen Tarnow, Ph.D.
                                                                            Avalon Business Systems, Inc.
                                                                            http://AvalonAnalytics.com

                                                                            The 2016 PARCC scores are out and they are quite interesting.  The zero score is 650 and the highest possible is 850, a 200 point spread. Below I show you the 2016 scores versus the 2015 scores in Algebra 1 for all schools.  There is a spread - the scores of those two years are not quite the same.

                                                                            Image:NJ PARCC Algebra 1 Scores Changes Over A Year

                                                                            If I plot the difference between the two years as a function of the 2015 score we end up with the plot below.  Overall the average scores have improved by 5, the standard deviation is 9, and the improvement is highest for the lowest performing schools.  Some schools improved by as much as 50 points but some schools declined by 40 points.  

                                                                            Image:NJ PARCC Algebra 1 Scores Changes Over A Year

                                                                            Some of these changes are statistical noise as can be seen in the corresponding plots of schools with at least 100 valid scores.  Again the improvement is on average 5 points but the standard deviation is a smaller 7.6.  The improvement is highest for the lowest performing schools.  Some schools improved by 35 points and some declined by 20 points.

                                                                            Image:NJ PARCC Algebra 1 Scores Changes Over A Year

                                                                            What could account for the large differences?  The students or the teachers?  Let's increase the number of valid scores to 200 to lower the noise from the student population further.  The corresponding plots are below. The improvement is on average 5 points and the standard deviation is 7.3 and the improvement is highest for the lowest performing schools.  Again some schools improved by as much as 20-30 points and some declined by 10-20 points.

                                                                            Image:NJ PARCC Algebra 1 Scores Changes Over A Year

                                                                            Since we varied the number of students and got similar results, it would seem that the difference in test scores are due to the teachers. In other words, variability between Algebra 1 teachers within the same school account for about 7 PARCC points (but can vary between -20 to 35 according to the plot just above).

                                                                            Finally, the extra 5 points are somewhat dependent on socioeconomic characteristics, see the plot below.  "A", "B" etc. refer to "District Factor Groups" and the order is low to high socioeconomic characteristics.

                                                                            Image:NJ PARCC Algebra 1 Scores Changes Over A Year

                                                                            The score differences as a function of socioeconomic characteristics are shown below:
                                                                            Image:NJ PARCC Algebra 1 Scores Changes Over A Year

                                                                            There is no strong relationship of large declines or improvements with socioeconomic characteristics, see below:

                                                                            Image:NJ PARCC Algebra 1 Scores Changes Over A Year

                                                                            As always, I reserve the right to be wrong.
                                                                            Comments Disabled

                                                                              NJ PARCC English 11 Scores Changes Over A Year: Differences Due to Socioeconomic Factors Minimized by Excluding Elite Students

                                                                              Eugen G Tarnow  January 4 2017 06:49:41 PM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com

                                                                              On September 9th, 2015 Commissioner Hespe allowed 11th grade students who were enrolled in AP or IB courses AND took the assessment associated with those courses a “waiver” from the PARCC grade 11 ELA assessment.

                                                                              The consequences of excluding the elite students from the PARCC test were large and strongly correlated with socioeconomic characteristics, see the plot below.  "A", "B" etc. refer to "District Factor Groups" and the order is low to high socioeconomic characteristics.  

                                                                              Image:NJ PARCC English 11 Scores Changes Over A Year: Differences Due to Socioeconomic Factors Minimized by Excluding Elite Students

                                                                              The waiver decreased the scores of high performing schools and increased the scores of low performing schools.  The score differences as a function of socioeconomic characteristics are shown below:

                                                                              Image:NJ PARCC English 11 Scores Changes Over A Year: Differences Due to Socioeconomic Factors Minimized by Excluding Elite Students

                                                                              The consequence can be seen the clearest in the graph below that shows the average scores by socioeconomic group for the two years plotted against each other.  The score differences in 2016 are only 44% of the differences in 2015.  It looks as if socioeconomic factors are only 44% of what they were before.

                                                                              Image:NJ PARCC English 11 Scores Changes Over A Year: Differences Due to Socioeconomic Factors Minimized by Excluding Elite Students

                                                                              Another way to put it: the F test for socioeconomic characteristics yielded 29 in 2015 but only 14 in 2016.

                                                                              As always, I reserve the right to be wrong.
                                                                              Comments Disabled

                                                                              Providers in Republican States Prescribe Higher Levels of Opiates Than in Democratic States

                                                                              Eugen G Tarnow  December 6 2016 09:13:23 PM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com


                                                                              AP wrote a story ( click here ) about how the makers of opioids lobby state legislatures very successfully.

                                                                              I found independently that Medicare opioid prescriptions are very strongly correlated with state poverty - the poorer the state, the more Medicare opioid prescriptions.  This presumably means that the state legislatures in poor states are easier to pay off than state legislatures in rich states.  The payoff results in opioid favorable state laws; a lack of laws limiting opioids.

                                                                              I then decided to look at whether there is a difference between parties: do states with Republican legislatures write less opioids than states with Democratic legislatures or vice versa?  

                                                                              It turns out that states with Republican legislatures have providers that write more Medicare opioid prescriptions. This is shown in the graph below.  In each of the graphs is displayed the percent Medicare opioid claims out of all Medicare claims as a function of median household income.  The red dots correspond to Republican governors and the blue dots to Democratic governors.  The lines are the least square fits to the dots.  

                                                                              In the top graph I find that states with Republican governors have more Medicare opioid claims than states with Democratic governors. In the middle graph we see that is even more so for Republican upper houses and, in the bottom graph, for Republican lower houses (I used the Wikipedia definition of lower and upper houses).

                                                                              Image:Providers in Republican States Prescribe Higher Levels of Opiates Than in Democratic States

                                                                              This result is not obvious.  Below are the same graphs for the murder rate.  It is a lot less sensitive to the median household income and the variation with party is not as consistent.

                                                                              Image:Providers in Republican States Prescribe Higher Levels of Opiates Than in Democratic States

                                                                              A one-way ANOVA showed that there was a significant difference between Democrat and Republican Lower Houses for opioids (F=16.7, p<0.001) but not for the murder rate (p=0.8), Upper Houses for opioids (F=4.6, p=0.015) but not for the murder rate (p=0.67), Governor for opioids (F=4.23, p=0.045) but not for the murder rate (p=0.58).
                                                                              Comments Disabled

                                                                              Do Math Supervisors Influence PARCC Scores?

                                                                              Eugen G Tarnow  November 19 2016 09:05:16 AM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com

                                                                              I have previously written about how difficult it is for a district to get good math teachers (as reflected in the weak correlation between district parental income and PARCC scores).  

                                                                              This time I asked the question: do math supervisors matter for the PARCC scores?

                                                                              If they do, then if the math supervisor is good, all the math scores would tend to be good and if the math supervisor is bad, all the math scores would tend to be bad.  In other words, there should be a strong correlation between the math scores of a school district if the math supervisor matters.  If the math supervisor does not matter, the math scores would be correlated with each other no more than with some other subject.

                                                                              I compared the correlations within the PARCC math scores with the correlations between the PARCC math scores and the NJBCT biology score.  I only included schools with complete scores, see http://njschoolperformance.com.

                                                                              The result was: no statistical difference between the two.  In other words, biology is as correlated with the math scores as math scores are correlated with each other.

                                                                              Thus it seems that math supervisors do not influence the PARCC scores.
                                                                              Comments Disabled

                                                                              Globalization helps unfree nations become free but free nations move the other way

                                                                              Eugen G Tarnow  September 21 2016 09:05:52 AM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com

                                                                              Here are two charts I find interesting in the midst of the discussion of globalization.  I used the Index of Economic Freedom (Heritage Foundation and the Wall Street Journal) to see what globalization does to free nations.  The left chart shows that GDP per capita tends to correlate strongly with freedom.  BUT the right chart shows that during the last three years, freedom did not correlate with the GDP ratio.  So at least in one measure globalization is not favoring free countries.

                                                                              Image:Globalization helps unfree nations become free but free nations move the other way

                                                                              The average freedom score, weighted by population, has increased by 0.7 points in the same time period, a positive sign.  However, freedom is moving from free to unfree countries: the difference in freedom correlates negatively with the freedom score.

                                                                              Image:Globalization helps unfree nations become free but free nations move the other way

                                                                              Something to discuss at this week's meeting of the United Nations General Assembly, no?

                                                                              And here is how we in the United States have lost our freedom since 2009:

                                                                              Image:Globalization helps unfree nations become free but free nations move the other way
                                                                               
                                                                              Comments Disabled

                                                                              The eight best school districts of NJ - a district for every budget

                                                                              Eugen G Tarnow  September 13 2016 02:09:05 PM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com

                                                                              When a family is looking to settle down in NJ they are often trying to figure out what is the best school district they can afford.  

                                                                              We ranked all the school districts based on their PARCC test data (http://NJSchoolPerformance.com ), including only school districts with complete data.  

                                                                              For every income level there is a highest ranking school district and here is our list:

                                                                              School district Avalon ranking Median family salary
                                                                              PHILLIPSBURG TOWN (Warren County) 62 $52,484
                                                                              JERSEY CITY (Hudson County) 54 $64,419
                                                                              FORT LEE (Bergen County) 46 $87,289
                                                                              NEW MILFORD (Bergen County) 17 $97,639
                                                                              LEONIA (Bergen County) 14 $100,283
                                                                              EDISON (Middlesex County) 12 $105,670
                                                                              MAHWAH (Bergen County) 4 $120,107
                                                                              BERNARDS (Somerset County) 1 $165,714



                                                                              Comments Disabled

                                                                              Educational Quality Control Lacking - New Jersey School District Subject Knowledge Test Results & Family Income

                                                                              Eugen G Tarnow  September 6 2016 09:58:15 AM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com

                                                                              We have seen that family income determines 45% of the Avalon school ranking of New Jersey school districts.  

                                                                              But income does not determine specific subject knowledge.  In other words, you do not get the bang for the buck you might expect in each subject.  

                                                                              In Figure 1 I show you how much of the subject knowledge is determined by the family income in NJ school districts.  Math knowledge increases from Algebra 1 to Algebra 2 perhaps because the Algebra 1 score is influenced by both middle school and high school students which might make the teachers difficult to supervise.  The trend is the opposite in English, presumably because parents are strongly involved in English 9 and then give up their involvement resulting in an almost random English score in grade 11.

                                                                              In other words, while district family income determines the AVERAGE score over all subjects, the SPECIFIC scores are left to chance.

                                                                              If you think school is important, it is time to get involved.  A talk with a low performing department supervisor might be a good beginning.

                                                                              Image:Educational Quality Control Lacking - New Jersey School District Subject Knowledge Test Results & Family Income
                                                                              Figure 1.  Importance of district income in determining subject knowledge.

                                                                              Comments Disabled

                                                                              New Jersey School Districts Rank Determined By Family Income But Bargains Still Remain to Be Had

                                                                              Eugen G Tarnow  September 5 2016 01:05:49 PM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com

                                                                              I performed probably the most comprehensive ranking of New Jersey school districts (see http://NJSchoolPerformance.com ).  Since SAT scores are determined by family income and less by the school performance, we ranked the New Jersey school districts based on the standardized test PARCC high school scores and the corresponding scores from middle school honors programs.

                                                                              I expected there to still be a correlation with family income but I was taken back when I found that family income made up 45% of the ranking (see figure below).

                                                                              Nevertheless, the remaining spread is large enough for there to be bargains to be had.  If you want to live in one of the top ten ranked school districts, expect to have to make about $103,000-$211,000 per year. (NOTE: the family income does not necessarily determine the amount of money the school get to spend per student - that's a different topic.)

                                                                              And if you live in school district that has low ranking despite being in a town with high family incomes - talk to the corresponding board of education (and good luck with that!).

                                                                              Image:New Jersey School Districts Rank Determined By Family Income But Bargains Still Remain to Be Had
                                                                              Figure. Avalon ranking of school district versus median family income.

                                                                              Comments Disabled

                                                                              NJ School Districts: SAT Scores, PARCC Scores, Suspension Rates & Family Income

                                                                              Eugen G Tarnow  August 26 2016 10:44:50 PM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com

                                                                              Many families live in particular New Jersey towns for the school districts and at least some select the school district based on the SAT scores. However, a higher SAT school district carries with it financial hardship.  One can make a more optimal financial decision if one could tease out what part of the SAT scores are based on school performance and what is based on individual family spending on SAT coaching – since the coaching can take place in any school district.
                                                                              For this purpose, I am able to exploit a new standardized test in NJ public schools, the PARCC test, which is relatively opaque to parents.  It was designed with a lowest possible score of 650, presumably to avoid parental shock, is accompanied by frequent assurances to students that the PARCC test scores are “confidential”, the student reports are almost unreadable (including a demographically matched set of school districts that do not match the local area) and the schools do not discuss their performance reports.  This suggests that the PARCC results reflect the combination of the quality of the school education and the quality of the students and less so parent involvement.  The SAT score, on the other hand, alerts parents and worries students as it carries great importance in college admission, suggesting that the SAT score reflects, in addition to the quality of the school education and the quality of the students, more student effort and parental input.

                                                                              One important part of the student effort and parental involvement to increase the SAT score is paid coaching.  It would seem reasonable that the access to paid coaching is income based.  The lowest incomes should correspond to little paid coaching while the highest incomes should correspond to the best paid coaching available. Thus if one removes the effects of the school environment via the PARCC scores, the remainder should reflect the effect of best paid coaching available (this reasoning is similar to that of Zwick & Green, 2007).

                                                                              Fig. 1 displays the effect of family income on PARCC and SAT scores as well as the student teacher ratio and the student suspension rate.  The student teacher ratio has a very small effect and will be neglected.  Income differences affect some subject scores more than others.  Note the trend for English – as high school progresses the correlation with income decreases.  The correlation with income is also low for Algebra 1. The strongest PARCC test correlation with income is the overall average.  The student suspension rate is also strongly correlated with income.
                                                                              Image:NJ School Districts: SAT Scores, PARCC Scores, Suspension Rates & Family Income
                                                                              Fig. 1 PARCC and SAT test score correlations with median family income.  


                                                                              In Fig. 2 is displayed the correlation with income of the SAT score with the PARCC average and the student suspension rates removed separately and together via a linear regression.  In Table 1. is displayed the SAT score difference for incomes between $50,000 and $170,000. The unaltered difference is 187 and the difference with the effects of the PARCC average score and student suspension rates is 74.

                                                                              Image:NJ School Districts: SAT Scores, PARCC Scores, Suspension Rates & Family Income
                                                                              Figure 2.  Correlation of SAT score with family income removing effects from suspensions and PARCC averages.


                                                                              Removing suspensions Removing PARCC Average Removing both PARCC Average and suspensions
                                                                              SAT score difference 50,000-220,000 income 187 (out of 800) 119 (out of 800) 91 (out of 800) 74 (out of 800)



                                                                              Table 1.  The difference in SAT scores for school districts with mean family incomes of $220,000 and $50,000.

                                                                              In Fig. 3 is displayed the student suspension rate as a function of income.  The functional form is close to exponential which means that an extra $30,000 in income results in the halving of the suspension rate.
                                                                              Image:NJ School Districts: SAT Scores, PARCC Scores, Suspension Rates & Family Income
                                                                              Figure 3.  The effect of family income on suspension rate is exponential; an extra $30,000 in family income lowers the suspension rate by 50%.


                                                                              In Table 2 is displayed the financial correlations of an SAT point.

                                                                              Increase in tuition per year Increase in earnings per year
                                                                              1 point increase in SAT score $36 $61



                                                                              Table 2.  The consequence of one SAT point.




                                                                              Discussion

                                                                              Many people live in particular New Jersey towns for the school districts and at least some select the school district based on the SAT scores.  I find that each SAT point correlates with an increase in the mean family income of about $1,000.  On the other hand, 74 points out of 800 are presumably attributable the cost of SAT coaching.  If the most expensive SAT coaching is $7,400 this is a lot cheaper alternative at a one-time cost of $100 per SAT point.  

                                                                              I find that college average SAT scores correlate with $61 additional yearly income per SAT point, partially offset by a tuition increase of $147 for a four year stay per SAT point, suggesting that SAT coaching is worthwhile (breakeven point occurs after four years) but a better neighborhood may not be worthwhile (Tarnow, 2015).

                                                                              Interestingly, the tuition increase for an SAT point is suboptimal for the colleges: for every point increase in the SAT score, a college can expect an increase in the median family income of $3,000 (in NJ, it may be different in other states).  However, they only tap $147.

                                                                              Colleges could use an “effective” SAT score with the role of family income removed but if they did, they would correspondingly compromise their financial needs.

                                                                              Within the PARCC test I find large variations in how much the median family income influences the subject scores (Fig. 1).  The influence is the lowest for Algebra 1 and 11th grade English. The reason is unclear though I venture to guess that it has to do with a lack of parental involvement: Algebra 1 is taught both in middle school and high school and may be harder to keep track of and 11th grade English may be a topic hard to understand and/or control.  I note that New Jersey is not a state in which parents are particularly involved with the schools other than comparing school districts.

                                                                              The suspension rate declines exponentially with median family income.  $30,000 halves the suspension rate.  The cost of moving from a “bad” neighborhood to a better one is extremely expensive.  Interestingly, teachers in the NJ school system can do this for free even though others can only do this at a substantial cost.  The freedom of teachers to move districts may also be a reason for low performing school districts staying low performing.  I suggest a solution to the problem: tax the contract benefit of moving to a richer school district at the difference in median family income between the two districts.
                                                                              Comments Disabled

                                                                              The Opium for the Poor Is – Opium. Providers in States with Low Income Prescribe High Levels of Opiates

                                                                              Eugen G Tarnow  August 26 2016 11:00:10 AM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com

                                                                              The current opiate epidemic has many reasons: pharmaceutical marketing, naïve and/or unscrupulous doctors, availability of cheap and cleverly sold Mexican heroin and inappropriate policy changes to encourage the idea that pain was something curable with drugs that were not addictive (Quinones, 2015).

                                                                              I find that another reason is lax supervision of Medicare providers.  I find that the Medicare opioid claims per Medicare recipient varies strongly by state.  It should not.  Medicare should have caught that but did not.  

                                                                              Interestingly the states with the largest number of claims are the poorest - with 51% of the variance coming from the median household income.

                                                                              Image:The Opium for the Poor Is – Opium. Providers in States with Low Income Prescribe High Levels of Opiates
                                                                              Comments Disabled

                                                                              NJ School Performance Reports are surprisingly interesting

                                                                              Eugen G Tarnow  August 19 2016 04:40:48 PM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com

                                                                              Last February the 2014-2015 school performance reports came out.  Here is a comparison between the high schools of two neighboring towns in New Jersey, Fair Lawn and Glen Rock.

                                                                              1.  Fair Lawn has not posted the report on its high school website for the last two years - the only link available is to the old 2012-2013 report which is announced as "now available".  Glen Rock does not post any link at all.
                                                                              2.  In English language arts/literacy the percentage who met or exceeded expectation was 57% in Fair Lawn and 42% in Glen Rock.
                                                                              3.  In Math the corresponding percentages were 43% (Fair Lawn) and 28% (Glen Rock).
                                                                              4.  The participation rate was 85% in Fair Lawn and 78% in Glen Rock, presumably indicating that the true difference in English and Math scores between Fair Lawn and Glen Rock may be larger.
                                                                              5.  Biology proficiency is 72% for Fair Lawn and 85% for Glen Rock.

                                                                              Let's think about that for a minute.  If Fair Lawn is generally better than Glen Rock in both English and Math, why should the Biology score be better for Glen Rock?  There seems to be a problem with the Fair Lawn Biology program.

                                                                              6.  The trend of meeting or exceeding expectations in English in Fair Lawn is downwards as students move from 9th to 11th grade: 60%, 54% and 36% in 11th grade.  Glen Rock has the same issue with a trend that goes 54%, 19% and 26%.

                                                                              The conclusion is that there seems to be a serious problem with the high school English program in both schools.

                                                                              7.  There is no such trend in Math, but the meeting or exceeding expectations in the Algebra 1 class of Glen Rock High School is a surprising 10%.  What is going on there?

                                                                              8.  The average SAT score was 1602 for Fair Lawn and 1744 for Glen Rock.

                                                                              Wait a minute.  Fair Lawn scores better in Math and English but when it comes to the SATs, Glen Rock wins hands down.  The SAT score seems to have nothing to do with the high school test scores!  Could it be that everyone in Glen Rock hires a tutor that is much better than its high school teachers?  Or do the Glen Rock students suddenly realize that they have to perform? How can Fair Lawn students copy that?
                                                                              Comments Disabled

                                                                              Big Data: If you can save 1% your are doing well - if you can save 30% run away! Response to Bloomberg’s Leonid Bershidsky

                                                                              Eugen G Tarnow  June 2 2016 09:10:03 AM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com

                                                                              My experience in big data has taught me one piece of "big data philosophy": if you can save 1% or so, you are doing great.  If you see a way to save 30% - run away as quick as you can!

                                                                              Before I explain the reason for this philosophy, let's use an editorial in Bloomberg by Leonid Bershidsky as background.  The title is "Big Data Is Still Only a Little Helpful and is quoting an article by Hyunyoung Choi and Hal Varian.

                                                                              Lenoid writes that trends in Google queries can predict the number of current autosales before official numbers are made available.  In other words, because Google calculates auto query reports much faster than official reports are put together on actual sales, the Google data can be helpful.  The predictions can be improved by 5-20% according to Choi and Varian but they are looking at second order differences.  The forecast without Google may be 6.34% and with Google trends 5.66% giving an improvement of 10.6% but the actual difference is less than one percent.

                                                                              If we consider the Choi and Varian improvement to be a 1% improvement, in my book they did a great job. People are pretty good at what they are doing and using a computer can make small improvements but it is hard to think of ways it can make a big difference on problems that have been studied many times over.

                                                                              Or can it?

                                                                              Sometimes it is easy to find much larger improvements using big data.  For example, I found that provider Medicare reimbursements are too high and they increase list prices by an average of 45%, that Medicare payments to hospitals for heart attacks could be lowered by 37% without effecting the quality of care.

                                                                              Why didn't Leonid Bershidsky write about these results instead and change the title to "Big Data Can Save 40%"?

                                                                              That is because if your big data algorithm finds such a large number it is likely it has encountered systemic corruption.

                                                                              Finding 30% in fields such as health care is like shooting fish in a barrel.  But then run away unless you are big data's equivalent of Dirty Harry.
                                                                              Comments Disabled

                                                                              Who is more eloquent - conservatives or liberals? The grade level of the Facebook political discourse.

                                                                              Eugen G Tarnow  May 18 2016 01:55:14 PM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com

                                                                              The Wall Street Journal recently devised a side-by-side comparison of what very conservative and very liberal users see when they go on Facebook.  

                                                                              I thought it might be interesting to see whose discourse level was more sophisticated - conservatives or liberals?  

                                                                              I used the comments that travel around with the news articles on all the topics categorized by the Wall Street Journal: "Hillary Clinton", "Donald Trump", "Bernie Sanders", "Barack Obama", "Guns", "Abortion", "ISIS", "Transgender" and "Facebook" and copied them into Word, replacing all names by "Jane" to remove any bias from "Donald Trump" having fewer characters than "Hillary Clinton", for example.

                                                                              I then ran the Readability statistics and found:

                                                                              Very conservative: 7.5

                                                                              Very liberal: 6.6

                                                                              using the Flesch-Kincaid Grade Level.

                                                                              I did not check what the error bars might be (probably a full grade) but note that our high school education seems under utilized for both our very liberal and very conservative friends.

                                                                              This entry is scored at the Flesch-Kincaid Grade Level 12.  

                                                                              The texts used are displayed below:

                                                                              Here's why Jane is APPEALING and ... Jane is APPALLING.
                                                                              Brave Navy Seal DESTROYS Jane Right To Her Face.
                                                                              This is awesome! We solute you soldier.
                                                                              Can we get 5,000 'LIKES' for this Seal?
                                                                              Jane's latest attack against Jane is a perfect example of why many Americans don't want her anywhere near the White House. This could be a BIG preview of Jane's strategy for the general election.
                                                                              Will these kind of attacks help Jane defeat Jane?
                                                                              Despite Jane's desperate attempts the facts of what really happened in the Benghazi attack are coming to light.
                                                                              The New York Times's hit piece on Jane continues to unravel. Now Jane says she has nothing but respect for Jane and they lied about her account too.
                                                                              ?I want to promulgate the idea that just because someone has a different opinion than yours it does not make them the enemy,? Jane said.
                                                                              "I?ve experienced more pure, unadulterated anti-Semitism since coming out against Jane?s candidacy than at any other time in my political career. Jane supporters...greeted the birth of my second child by calling for me, my wife, and two children to be thrown into a gas chamber."
                                                                              Want nightmares?
                                                                              WATCH:
                                                                              He literally lost delegates with no one else running.
                                                                              HEY SOCIALIST Jane FANS: CHECK OUT THE SHOCKING VIDEO OF SOCIALISM IN VENEZUELA
                                                                              This family is not fit to run a lemonade stand let alone a country.
                                                                              MSNBC?s Nicolle Wallace argued that if Jane loses to Jane in November it will be because Jane stayed in the presidential race so long and due to his effective attacks on the Democratic front-runner.
                                                                              Jane supporters -- are you getting this?
                                                                              "As the young American left continues to champion the rise of socialist policies in our own country, many are left wondering if these idealistic collectivists are really prepared for the realities of socialism." -- Generation Opportunity
                                                                              SHOTS FIRED INTO Jane NEVADA HEADQUARTERS!
                                                                              Public schools in Jane's hometown of Chicago have just taken this transgender bathroom insanity to the next level.
                                                                              What they're demanding students do now shows exactly how set they are on brainwashing our children that perversion is perfectly normal.
                                                                              This is sick, and it must be stopped NOW.
                                                                              This is not OK!
                                                                              MUSLIMS TAKING OVER A SMALL TOWN IN TENNESSEE In Jane's Plan For A "New America" [Video]
                                                                              Uh...
                                                                              What treachery. It it the greatest acts of treason by a President in American history (rivaled by the Benghazi betrayal and cover-up) and yet not a critical word in the press. Scant attention is paid to Jane's towering betrayal.
                                                                              How many of you agree she should be headed for prison instead of the White House?
                                                                              "People who really are struggling against oppression do not resort to inventing slurs or smelting smoking guns; they have enough to deal with in the real world."
                                                                              Who could have ever predicted this would happen?
                                                                              Jane was molested by male migrant gang in New Year attacks. The 17-year-old and her friend described it as 'the worst night of our lives.' She said worst part was witnesses, including police, did nothing to stop it.
                                                                              These girls need guns just like the Jews needed guns -- Germany is reverting but this time the victims are Germans.
                                                                              NYT & The Left are desperately trying to make Jane look like a cad.
                                                                              Why don't they turn those guns of The Lolita Express's frequent flyer and serial rapist, Bill Jane?
                                                                              ACLU Trying to Shut Down Catholic Hospitals Because They Refuse to Do Abortions
                                                                              Yesterday's Supreme Court ruling on President Jane's abortion-pill mandate was a major victory not only for the Little Sisters of the Poor, but also for the religious freedom of every American.
                                                                              Read and Share as we explain the Court's ruling and why it is significant.
                                                                              Louisiana Legislature Passes Bill to Ban Dismemberment Abortions Tearing Off Baby?s Limbs
                                                                              It's time to end taxpayer funding for abortion providers. Click here!
                                                                              BREAKING: South Carolina Legislature Passes Bill to Ban Abortions After 19 Weeks When Babies Feel Pain
                                                                              ***ALERT***
                                                                              JANE was right again.........
                                                                              At Least Nine Americans Who Joined ISIS Were Immigrants to US!
                                                                              CARTEL OR ISIS? TEXAS FATHER Pushes For Answers In Son's Murder: "They cut my son's head off and they took my son's head!"
                                                                              And it's crickets from the main stream media on this horrific crime...
                                                                              63 confirmed deaths!!
                                                                              Here's why Jane is APPEALING and ... Jane is APPALLING.
                                                                              These Target bosses and employees were just starting to get comfortable with the company's sick transgender bathroom policy.
                                                                              But then they looked outside and saw who was waiting for them in the parking lot, and that changed everything.
                                                                              Now they're as worried as they've ever been. Good. We need to keep up the pressure until they remove this perverted rule once and for all.
                                                                              Target has committed itself to keeping its transgender bathroom policy in place, and now many of its stores are going to be facing HARSH consequences. Executives at Target never expected this to happen, and it may cause the company A LOT of damage.
                                                                              Share this if you are boycotting Target!
                                                                              Public schools in Jane's hometown of Chicago have just taken this transgender bathroom insanity to the next level.
                                                                              What they're demanding students do now shows exactly how set they are on brainwashing our children that perversion is perfectly normal.
                                                                              This is sick, and it must be stopped NOW.
                                                                              OUR GAY PRESIDENT SPEAKS: If being transgender is now considered a human right, then where does it end? When people want to have sex with animals will that become a human right? Does the North American Man Boy Love Association qualify for protection under Jane?s madness? If ?all love is love?, as Jane and the LGBT Mafia keep telling us, then NAMBLA should absolutely come under Federal protection.The bottom of this pit runs very deep, and it ends in Hell.
                                                                              Jane Appoints Mentally Sick Transgender Man to Lead Presidential Council
                                                                              They probably won't let you read this, but we'll try anyway....
                                                                              Will Facebook's Mark Zuckerberg sit down, man to man, with Milo Yiannopoulos?
                                                                              Please take a moment and go to the website and check out ANY article as Facebook is playing games with our algorithm again. Thank you!
                                                                              The liberal media are ignoring a MAJOR part of the Facebook scandal.
                                                                              Facebook was recently caught red-handed suppressing conservative news and viewpoints.
                                                                              According to this liberal at the New Republic, free speech SHOULD be stifled if it comes from conservatives. Do you agree with him?
                                                                              Be sure to visit us at and like us at



                                                                              Yikes.
                                                                              How many people need to say Jane is innocent before Republicans drop it? Never mind. We know they won't.
                                                                              Jane bested Jane in dramatic fashion.
                                                                              HISTORY =>
                                                                              Jane is the winner of an astonishingly close race against Jane in Kentucky, NBC News projected on Tuesday night.
                                                                              What an idiot.
                                                                              Just one of the many ways he respects women.
                                                                              Let's be real. Who wants a President Jane?
                                                                              British Prime Minister Jane brilliantly explains why they shouldn't ban Jane.
                                                                              Video by Occupy Democrats, LIKE our page for more!
                                                                              The author wanted one page he could send social media "trolls" who deny Jane's anti-gay, racist, sexist rhetoric ? and insist you "prove it."
                                                                              We were only too happy to oblige.
                                                                              "It's sort of painful to see a good person like Jane turned into such a sullen and resentful man. And doubly painful to see him take his followers down that path too."
                                                                              Making the case for energy and enthusiasm's importance, Jane tells Californians it ain't nearly over yet.
                                                                              "I come from the working class of this country and I'll be damned if we will allow the Republican Party, whose job is to represent the rich and the powerful, to win the votes of working-class Americans." ?Sen. Jane
                                                                              Jane notches another victory.
                                                                              TYT Politics Reporter Jane reported from Senator Jane Portland, Oregon headquarters on May 15th, 2016. Inside lies walls draped with incredible artwork depicting the progressive Senator and other themes of the progressive movement.
                                                                              BREAKING ?
                                                                              What an idiot.
                                                                              This disproves everything Republicans have been saying about raising wages.
                                                                              This disproves everything Republicans have been saying about raising wages.
                                                                              Jane hates aiding the sick, apparently...
                                                                              LIKE A BOSS! Pres. Jane Makes Overtime Pay Available To 4 Million Americans. Read more on the rules issued today by POTUS.
                                                                              Grio fam, what are your thoughts on this?
                                                                              I wonder how he's going to explain that mustache in prison? He had lots guns and ammo on him, too.
                                                                              Doesn't this sound kind of familiar? Isn't this the same fear-mongering the GOP tried when President Jane ran?
                                                                              Something is terribly wrong with America, babies, and guns. Don't know what the solution is, but this is officially way past insane.
                                                                              What is he smoking?! I guess he was just in Washington state.
                                                                              AC/DC have released a video showing rehearsal footage ahead of their live debut with temporary frontman Guns N' Roses' Jane. Watch below.
                                                                              ?It wasn?t abortion that birthed the religious right. It was good old white Nativism and anti-government anger when the IRS challenged evangelicals? god given right to go to school without black people.?
                                                                              He is totally busted.
                                                                              For over 30 years, Jane has been dedicated to keeping the clinic doors open to ensure women of Alabama have access to reproductive healthcare that includes abortion care services. Last year she was one of the recipients of the ACLU?s the Jane Medal of Liberty because she is a dedicated nurse, mother, provider and hero.
                                                                              Alabama Governor Jane has signed into law a measure banning abortion clinics from operating within 2,000 feet of a K-8 public school.
                                                                              Isn?t it ironic: tough abortion restrictions do not lead to fewer abortions? Restrictions do not prevent abortion, but they do make it less safe. As abortion restrictions sweep across the country, new reports show that such restrictions don?t actually decrease the abortion rate.
                                                                              This week ISIS has stepped up violent attacks in Iraq ? but you might have missed it.
                                                                              WARNING: Graphic footage.
                                                                              This is proof that Republicans don?t care about ISIS unless they?re using it to scare up votes.
                                                                              These are outright lies. And Jane just keeps repeating them.
                                                                              What's the difference between Jane and ISIS? It's starting to sound like not much.
                                                                              The war against ISIS is being conveyed to us as if we're on Snapchat. Important things happen, they appear in front of us and then... POOF!... they're gone. But just because we now have terrifyingly short memories does not, however, mean that critical things did not happen.
                                                                              A stark contrast on LGBTQ issues.
                                                                              Being transgender is not a mental disorder ? so why is it still classified as one?
                                                                              This man doesn?t belong on a public platform. He belongs in an institution.
                                                                              She donated her hair to kids with cancer, then she met a bigot in a bathroom.
                                                                              Read the whole story HERE
                                                                              MISSING: 19-year-old Jane was last seen leaving a dorm party on campus. She was on a full scholarship at Columbia University in the City of New York.
                                                                              The student's mobile phone & bank account numbers have been changed. Her Facebook page has been deactivated as well. DETAILS ---
                                                                              Jesus is going to be so disappointed in him.
                                                                              Um...
                                                                              Last November, she posted a message on Facebook saying she had taken an overdose.
                                                                              This is the result of anti-LGBT hate speech..


                                                                              Comments Disabled

                                                                              The 2008 mortgage crisis in clusters

                                                                              Eugen G Tarnow  May 16 2016 09:20:13 AM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com

                                                                              I noted in a previous posting Fannie Mae mortgages & credit scores that it appeared that the credit score distribution for the 2007 Fannie Mae loans had two peaks.  

                                                                              One can, in fact, see clustering in the credit scores.  Interestingly, there are also two clusters in 2005, 2003 and even 2001!  

                                                                              What do we know about those clusters?

                                                                              The most important definer of the clusters is the average credit score of the borrower:

                                                                              Image:The 2008 mortgage crisis in clusters

                                                                              Notice how the clusters are far apart before the mortgage crisis.  The largest difference occurs in 2007 in which the difference is 102 points.  
                                                                              Yet the loan amounts to these two clusters are almost the same:

                                                                              Image:The 2008 mortgage crisis in clusters

                                                                              The smallest difference in loan amounts occurs in 2003 in which it is 1.8%.  How can any borrower decide to loan the same amount to two populations that differ by 100 points in the credit score?

                                                                              Before the mortgage crisis the number of borrowers in the high score cluster goes down to 56% and after the mortgage crisis it goes up to 72%:

                                                                              Image:The 2008 mortgage crisis in clusters

                                                                              Beyond the credit score, the clusters also differ by the proportion single borrowers:

                                                                              Image:The 2008 mortgage crisis in clusters

                                                                              In the low rated cluster it starts out at 5% higher in 2001 but it keeps increasing even through the mortgage crisis.  In 2013 the difference was 13%:

                                                                              Image:The 2008 mortgage crisis in clusters

                                                                              The interest rates are barely different for the two groups:

                                                                              Image:The 2008 mortgage crisis in clusters

                                                                              From the data available to me I am not able to discern the composition of these clusters.  

                                                                              But it is curious that they exist at all.
                                                                              Comments Disabled

                                                                              The Lift of An Alzheimer’s Memory Test

                                                                              Eugen G Tarnow  May 2 2016 04:58:45 PM
                                                                              The proportion of old people is increasing and with that increase follows an increase in particular types of disease, one of which is Alzheimer’s disease.  If Alzheimer’s disease is not an example of Gompertz’ law (an exponential death rate purely due to age) it might be possible to design drugs that may slow down or even reverse the disease progress.

                                                                              In order to put those drugs to clinical trials, symptoms of Alzheimer’s disease have to be easily measurable.  The “gold standard” of Alzheimer’s symptoms, unfortunately, are the results of an autopsy.  Thus “silver” standards have to be used.

                                                                              One of the silver standard symptom categories are memory failures.  Alzheimer’s disease attacks the brain and causes damage to the memory.  There is some reason to believe that the damage follows a particular path in the brain and the memory failures would then follow a well-defined path.  It is believed, for example, that the diagnosis of “mild cognitive impairment” represents early Alzheimer’s disease.

                                                                              While it is unlikely that damage to long term memory could be reversed by drugs (even if the brain tissue comes back the information is gone), damage to long term memory may be slowed and damage to short term memory may even be reversible.

                                                                              But what is short term memory?  Despite what is commonly believed this question is not settled.  Accordingly, there are many Alzheimer’s short term memory tests.

                                                                              One of the memory tests most sensitive to Alzheimer’s disease is “free recall” in which a set of words are studied and then repeated.  Unfortunately, words can be “chunked” together, creating larger meaningful units that each are as easily remembered as a single word.  Some of us can chunk words more easily than others so that a word free recall test of ten words can appear to some as just five “chunks”.  Thus test results cannot be compared between individuals and instead would require multiple tests over time to detect individual changes.  

                                                                              We at Avalon Analytics wanted to see whether an extremely well defined memory test, we named TUT (Tarnow, 2013), with items that lack meaningful connections would be sensitive to Alzheimer’s disease.  The test items were three double digit integers (the capacity of “working memory” is about 3), the particular numbers selected to make chunking very difficult.  Since the test items were common we thought that they may not be sensitive to education or language.  To lower the statistical noise level the TUT test consisted of three sets of three numbers each.

                                                                              We were able to work with the Alzheimer’s Disease Research Center at Icahn School of Medicine at Mount Sinai (ADRC ISMMS).  The 132 participants (Table 1) were part of a clinical study for aging and dementia. Inclusion criteria included 65 years of age or older, primarily English/Chinese/Spanish speaking, visual and auditory acuity adequate for cognitive testing, willingness to participate in all clinical assessment, and having a study partner available as an informant.  Three records indicating dementia due to oncology treatment and vascular dementia were discarded as was a record of a subject who refused to recall more than one number.

                                                                              DX
                                                                              Frequency
                                                                              Percent
                                                                              Valid Percent
                                                                              Cumulative Percent
                                                                              Valid Normal
                                                                              83
                                                                              62.9
                                                                              63.8
                                                                              63.8
                                                                              MCI
                                                                              37
                                                                              28.0
                                                                              28.5
                                                                              92.3
                                                                              AD
                                                                              10
                                                                              7.6
                                                                              7.7
                                                                              100.0
                                                                              Total
                                                                              130
                                                                              98.5
                                                                              100.0
                                                                              Missing System
                                                                              2
                                                                              1.5
                                                                              Total
                                                                              132
                                                                              100.0



                                                                              Table 1.  Breakdown of study subjects.

                                                                              We found that the participants remembered on average 2.6 items, independent of language (English, Cantonese, Mandarin and Spanish), independent of gender, age and education.

                                                                              The participants were diagnosed as normal, Alzheimer’s disease, and two types of MCI (“mild cognitive impairment”).

                                                                              The test detected participants with Alzheimer’s disease (remembered an average of 1.9 items versus 2.6 for normal participants) but was insensitive to participants with MCI (remembered an average of 2.5 items).  The standard deviation for both Alzheimer’s disease and normal subjects were 0.6 items.  The probability of a subject in the test population having Alzheimer’s disease as a function of the number of items remembered is displayed in Figure 1.

                                                                              The lift is calculated by taking the ratio of the highest target response of 40% from Fig. 1 and the average response (the prevalence of Alzheimer’s disease in the sample) of 7.6%.  This gives a lift of 4 at 20% (the lift below 1 is 0!).

                                                                              If we search for a meaning of the result of our very simple memory test it is quite complex.  One of the participants with Alzheimer’s disease had a perfect score on our test.  Thus the loss of working memory capacity for double digits is a common but not necessary condition for Alzheimer’s disease (clinical diagnosis, not autopsy).  Perhaps there are separate working memories for various item categories?  Or is the memory system with the worst symptoms geographically close in the brain to where the working memory for double digits is?  In addition, the participants with the lowest working memory capacity for double digits did not have dementia.  How is it possible to function with such an impaired memory?  Perhaps double digits are just not that important?

                                                                              Time will tell.

                                                                              Image:The Lift of An Alzheimer’s Memory Test
                                                                              Fig. 1. Probability that a subject coming to the clinic will have AD as a function of the TUT 3 item recall.

                                                                              Bibliography:

                                                                              Tarnow, E. (2013). U.S. Patent Application No. 14/066,195.

                                                                              Tarnow, E. (2016). Preliminary Evidence -- Diagnosed Alzheimer's Disease but Not MCI Affects Working Memory Capacity - 0.7 of 2.7 Memory Slots is Lost. arXiv preprint arXiv: 1603.07759.

                                                                              Biography:

                                                                              Eugen Tarnow is a Director of Data Science at Avalon Analytics (see http://avalonanalytics.com ).  He has a PhD in physics from MIT and have published research in a variety of fields including surveys, marketing, air plane cockpit behavior, and human memory.

                                                                              He can be reached at etarnow@avabiz.com
                                                                              Comments Disabled

                                                                              Employers are unhappy with your education - so take the PARCC!

                                                                              Eugen G Tarnow  April 7 2016 09:56:09 AM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com

                                                                              I just went to hear a presentation about why my child should be participating in the PARCC testing by Donna Egreczky of the NJ Chamber of Commerce.

                                                                              Here data included:
                                                                              • 49% of employers dissatisfied with high school students entering workforce
                                                                              • 76% of college instructors report students were not prepared at all for college expectations
                                                                              • 76% of college students report SOME gap in overall preparedness.  47% report LARGE gaps.
                                                                              • Armed forces reject 75% of applicants
                                                                              • College degree not predictive of career success
                                                                              • High grades in high school not predictive of college degree attainment

                                                                              This is a fascinating "big data" analytics problem.  

                                                                              And the devil is in the details.

                                                                              I do not think participating in the PARCC testing is the "nugget" that will lead to improvements in the data she presented.

                                                                              Rather, it is to do a knowledge census of the working population every ten years, see what of this can be taught in grade school and stop the mandatory education in 8th grade.  After all, the working population, on average, probably functions on the 8th grade level just fine.  And the school curriculum set after World War II is not relevant anymore.  And neither high school nor college teach "critical thinking" - perhaps it cannot be taught at all.
                                                                               

                                                                              Comments Disabled

                                                                              Congratulations to Rainer Weiss for discovering gravity waves!

                                                                              Eugen G Tarnow  February 11 2016 01:18:49 PM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com

                                                                              When I was an undergraduate at MIT, it was hard to get into a research group because as an undergraduate you are pretty useless!  

                                                                              The first group that accepted me was the one headed by Rainer Weiss, a professor that reminded me of Groucho Marx.  Instead of a cigar he had a pipe.  

                                                                              Rainer Weiss really cared about his students.  I found out that I was not suited for experimental physics after almost blowing up a container of liquid helium  a couple of times.  He became my advisor nevertheless and in the end I got to do some theoretical research and went on to do a PhD in physics.

                                                                              Back then Rainer Weiss proposed a way to detect gravity waves and kept trying to get the project going.  This was 35 years ago.  He did get it going and it was just announced that he succeeded.

                                                                              I can't think of anyone more deserving!
                                                                              Comments Disabled

                                                                              Estimating the cost of your hospital

                                                                              Eugen G Tarnow  February 9 2016 03:28:24 PM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com

                                                                              Hospital charges can vary widely - the same visit can result in different number of charged items and different charges for each item.

                                                                              So we created CostOfHospitals.com,  a website which estimates hospital charges (using our own algorithm).  We do not guarantee results but hope it might be helpful for both those with insurance (some hospitals may charge more items per visit) as well as those without insurance.
                                                                              Comments Disabled

                                                                              AVVO lawyer ratings are statistically unusual

                                                                              Eugen G Tarnow  February 9 2016 11:23:16 AM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com


                                                                              Internet ratings are a fascinating topic (I have covered drug ratings before - see Internet drug ratings: AskAPatient.com & WebMD correlate well but Drugs.com does not).  Yelp has recently been in the news with its ratings.  AVVO rates lawyers in a controversial way and has been taken to court but they won.

                                                                              I have been interested in AVVO ratings because it is in AVVO's interest to rate lawyers who do not use them artificially low and then increase their ratings when the lawyers "claim their profiles," i.e. start to work with AVVO and perhaps pay for their listings. If this is part of their business model one would find statistical properties of the ratings that deviate from what would be the norm.

                                                                              This entry shows that there is an unusual distribution of AVVO lawyer ratings.  

                                                                              Typically, a rating scheme would yield a normal curve - i.e. one peak in the middle and slow decrease from that in both directions.  I did a Google search on AVVO ratings today, and found the following distribution for ratings 6.1 and above (the AVVO ratings go up to 10.  I avoided the ratings below 5 because in my Google search the consumer ratings (5 and below) interfered with the AVVO ratings):


                                                                              Image:AVVO lawyer ratings are statistically unusual
                                                                              Figure 1.  AVVO ratings of lawyers.


                                                                              There are very large peaks at 6.5, 6.7 and 8 with hardly any lawyers in-between.  The coveted 10 out of 10 spot has a peak but there are almost no ratings from 8.9 to 9.9.

                                                                              AVVO claims their ratings represent the quality of lawyers according to the following schema:

                                                                              10.0 - 9.0 Superb
                                                                              8.9 - 8.0 Excellent
                                                                              7.9 - 7.0 Very Good
                                                                              6.9 - 6.0 Good

                                                                              If that is true the ratings should obey a normal distribution.  There appears to be many lawyers that are "good" and many that are on the bottom of the "excellent" category but hardly anyone who belong in the "very good" category.  And in the superb category, basically everyone is at the top of the category.  All this makes the AVVO ratings statistically highly unlikely.  Or there is something wrong with my Google search terms.

                                                                              If the business model is based on giving non-cooperating lawyers low ratings and cooperating lawyers high ratings one would expect that higher peaks would belong to cooperating lawyers while lower peaks would belong to non-cooperating lawyers.

                                                                              Indeed, the first five entries for lawyers rated 8.0 had pictures and their profiles had evidently been claimed by the lawyers.  In the first five entries for lawyers rated 6.7, 2 lawyers had not claimed their profiles.  In the first five entries for lawyers rated 6.5, however, all the profiles had been claimed.

                                                                              I decided to email one of the 6.5 rated lawyers to find out why he is rated low and found that the AVVO site immediately suggested other lawyers to write to as well.  I decided to try the same with a 10 rated lawyer and no other lawyers were suggested.  That suggests that lawyers with 10 ratings is getting better AVVO services than lawyers with 6.5 ratings.  That might mean that lawyers with 10 ratings pay AVVO more.

                                                                              This may be a measure of the number of high quality customer AVVO has.  If so this number is 5800.

                                                                              My investigation only shows that the rating distribution is likely very unusual, not that there is a direct link between payment and ratings.  

                                                                              Thus it is possible to rate the AVVO rating on statistical plausibility.  We give them 5
                                                                              stars out of a possible 50 (note that Google can't figure this out and rated this blog accordingly so we changed the rating to a 5...).
                                                                               
                                                                              Comments Disabled

                                                                              Comparing holy texts? How the Bible looks different in two different translations

                                                                              Eugen G Tarnow  February 4 2016 09:55:22 AM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com

                                                                              It is tempting to use Natural Language Processing to compare holy texts across religions as was recently done by a colleague of Avalon Analytics.  But there is a need to be careful because there is much riding on the details.

                                                                              Below I show you what a word frequency comparison looks like of the SAME text (Bible from http://unbound.biola.edu/index.cfm?method=downloads.showDownloadMain ) but in different versions.  The top panel displays the "Basic English" version and the bottom panel displays the "American Standard" version.  To make the comparison nothing has been done to the text except removal of "stopwords" (words that are too common to convey understanding.  List is taken from the NLTK library).

                                                                              I note two severe discrepancies.  In one version "shall" is extremely important but in the other it is nowhere to be found; in one version "give" is important but in the other version - it can't be found!

                                                                              Image:Comparing holy texts? How the Bible looks different in two different translations
                                                                              Image:Comparing holy texts? How the Bible looks different in two different translations
                                                                              Figure 1.  Word frequency plots of the Bible.  Top panel is the Basic English version, the bottom panel displays the American Standard version.

                                                                              Comments Disabled

                                                                              Medical device deaths do not attract attorneys?

                                                                              Eugen G Tarnow  February 3 2016 12:20:23 PM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com

                                                                              There is a new tool for class action attorneys specializing in medical device suits: http://fdafeed.com .  That site displays manufacturers together with the number of death reports and attorneys mentioned.  The data is taken from an FDA database.  

                                                                              Interestingly there is not much of a correlation between the number of death reports and number of attorneys (see Figure 1).  Is it because most deaths do not lead to possible legal claims or is it because attorneys are not yet using big data?


                                                                              Image:Medical device deaths do not attract attorneys?
                                                                              Figure 1.  Number of death reports versus number of attorneys for medical device adverse events.

                                                                              Comments Disabled

                                                                              Pew Research creates something out of nothing: fooling Financial Times and Naked Capitalism with a particular definition of "Middle Class"

                                                                              Eugen G Tarnow  December 14 2015 09:14:05 AM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com

                                                                              D'Israeli, former Prime Minister of England, coined the phrase "Lies, damned lies and statistics".  And Pew Research is a good example, with Financial Times and Naked Capitalism on the band wagon.

                                                                              According to CNN, there are five ways to define "Middle Class": income, wealth, consumption, aspiration and demographics.

                                                                              Pew Research decided to use income as its definition.  Within income, CNN includes several ways to define middle class: middle fifth, middle three fifths and Pew's definition ranging from 2/3 to 2 times the median income.

                                                                              Pew's headline screams out: "The American Middle Class Is Losing Ground" leaking members to those "in economic tiers above and below it".  The Financial Times writes "America’s Middle-class Meltdown: Core shrinks to half of US homes" and Naked Capitalism concludes: "Demise of the US Middle Class Now Official".

                                                                              If Pew would have chosen a different income based definition of the middle class: the middle fifth or the middle three fifth in income, they would have had no news...

                                                                              A comment on Naked Capitalism went unpublished despite its motto of "fearless commentary" - they apparently feared this one.
                                                                              Comments Disabled

                                                                              Pew, Financial Times & Naked Capitalism get fooled by statistics - claim the middle class is disappearing is simply a shift to higher incomes

                                                                              Eugen G Tarnow  December 11 2015 02:31:45 PM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com

                                                                              Sometimes I fall off my chair when I see statistical incompetence.  This time Pew Research, Financial Times & Naked Capitalism all fell victims to their own stupidity.

                                                                              The claim is that over time the middle class is disappearing and that this is shown by data from the Current Population Survey.  These two graphs are shown in Naked Capitalism (
                                                                              http://www.nakedcapitalism.com/2015/12/demise-of-the-us-middle-class-now-official.html ):

                                                                              Image:Pew, Financial Times & Naked Capitalism get fooled by statistics - claim the middle class is disappearing is simply a shift to higher incomes

                                                                              Image:Pew, Financial Times & Naked Capitalism get fooled by statistics - claim the middle class is disappearing is simply a shift to higher incomes
                                                                              Fig. 1.  House hold income distributions in 1971 and in 2015 (or should it be 2014?) from Naked Capitalism.

                                                                              And the Naked Capitalism commentary goes "The Financial Times headline is uncharacteristically dramatic: America’s Middle Class Meltdown: core shrinks to half of US homes. And I find their infographic that charts the the shrinkage of the middle class cohort over time to be more informative than the Pew charts that presented the same information; we are partially replicating it by showing the starting and end shots:"

                                                                              Really?  Incomes obey what is called a "lognormal" distribution because they are always positive.  Below I show you what happens when the average of a log normal distribution increases (yes, people make more money over time!).  The blue distribution has an average "income" of 1 and the red distribution has 60% higher average "income".  Indeed, it looks like the "middle class" disappears but it is just a function of how a lognormal distribution changes its look when the average increases.  And the +200k spike increase in the graphs of naked capitalism is there too.  Of course, if the incomes increase, then the spike, which is the sum of all incomes over the scale of the graph, has to go up a lot...

                                                                              Image:Pew, Financial Times & Naked Capitalism get fooled by statistics - claim the middle class is disappearing is simply a shift to higher incomes
                                                                              Fig. 2.  Changes in the "look" of a lognormal distribution as the average increases.  At 10, the point represents the sum of all values above 10.

                                                                              If I re-graph Fig. 1 on a linear and a log scale this shift is more apparent:

                                                                              Image:Pew, Financial Times & Naked Capitalism get fooled by statistics - claim the middle class is disappearing is simply a shift to higher incomes

                                                                              A comment on Naked Capitalism went unpublished despite its motto of "fearless commentary" - they apparently feared this one.
                                                                              Comments Disabled

                                                                              Sales people do not care about analytics!?

                                                                              Eugen G Tarnow  December 3 2015 08:47:16 AM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com

                                                                              One would think the people closest to the money in an organization would care the most about using analytics.  Data from application sales at Salesforce.com seem to invalidate this assumption.

                                                                              While the actual sales of AppExchange apps is not public information, the popularity and number of ratings are displayed.

                                                                              The average number of ratings of the overall top 20 pay applications is 584 while the corresponding number for analytics apps is only 14, 2.3% of the total.

                                                                              Image:Sales people do not care about analytics!?

                                                                              The question is why?  Is it that sales people do not want to be supervised or that their task is so socially complex that a computer app cannot be helpful?
                                                                              Comments Disabled

                                                                              An Inexpensive Way to Measure Loci of Control: Counting Will, Must & Should

                                                                              Eugen G Tarnow  November 4 2015 05:19:40 PM
                                                                              By Eugen Tarnow, Ph.D.
                                                                              Avalon Business Systems, Inc.
                                                                              http://AvalonAnalytics.com

                                                                              One of the famous personality tests in psychology, the Rorschach ink blot test, yields a lot of interesting data.  One of them may surprise: the locus of control.  The locus of control is a way to say whether you feel that you control your own destiny or that you feel that your destiny is controlled by others.  

                                                                              The cost of a Rorschach ink blot test is in the thousands of dollars.  Might there be a cheaper way to do it?

                                                                              I suggest that one might be able to estimate the locus of control from a person's writings. Here is a pilot study that focuses on the use of three words: "will", "must" and "should".  "Will" corresponds to an internal locus of control and "must" corresponds to an external locus of control.  "Should" expands the definition of the locus of control to include a vector that can point either towards or from the person.  Of course, the meanings of the three words overlap to some extent but for now, let's go with these assumptions

                                                                              So where are authors' loci of control in the common Gutenberg library?  In Fig. 1 we find that the authors with the least number of "wills" are Austen, Blake and Carroll.  The book with the most "wills" is the Bible.  The author with the most "shoulds" is Blake.  

                                                                              If we take the difference between "wills" and "musts" to find the book with the strongest internal locus of control (see Fig. 2) it is the Bible. This makes sense since it is written primarily from the point of view of God.  Why Austen and Carroll are the authors with the strongest external loci of control can be a college course in itself, but seems plausible.

                                                                              To summarize, I have presented a new way to measure the locus of control by counting the relative frequency of the words "will", "must", and "should".


                                                                              Image:An Inexpensive Way to Measure Loci of Control: Counting Will, Must & Should
                                                                              Fig. 1. The number of "wills", "musts" and "shoulds" in the Gutenberg library.


                                                                              Image:An Inexpensive Way to Measure Loci of Control: Counting Will, Must & Should
                                                                              Fig. 2. The number of "musts" subtracted from the number of "wills" in the Gutenberg library.

                                                                              Comments Disabled

                                                                                Dr. House’s Vicodin and Sheriff Longmire’s Zoloft criminals - how well did pharma do?

                                                                                Eugen G Tarnow  November 3 2015 10:29:56 AM
                                                                                By Eugen Tarnow, Ph.D.
                                                                                Avalon Business Systems, Inc.
                                                                                http://AvalonAnalytics.com

                                                                                A very famous social psychologist, Stanley Milgram, once tried to find out whether violence on TV translates into violence in society.  His findings were that the crime rate did not go up after a crime was shown on TV, BUT that the type of crime tended to imitate the TV show.

                                                                                Recently it was found that the number one drug for Medicare patients is Vicodin, the drug that gets Dr. House hooked on TV.  Did the TV show influence the number of prescriptions of vicodin?  If so, in which direction?

                                                                                We don't have the resources to do this project, but someone ought to do it.  Help has been offered on youtube: https://www.youtube.com/watch?v=PSaasBjeJek

                                                                                One would need to get the daily prescription data from IMS and then the show data (from Nielsen?).

                                                                                Once that is done one can turn one's attention to the Longmire show's Zoloft criminals - who desperately want to make Zoloft available to prevent suicide of veterans (not much evidence that would work).  Did the Zoloft prescriptions go up?
                                                                                Comments Disabled

                                                                                The Death Spiral of health insurance

                                                                                Eugen G Tarnow  November 2 2015 12:50:15 PM
                                                                                By Eugen Tarnow, Ph.D.
                                                                                Avalon Business Systems, Inc.
                                                                                http://AvalonAnalytics.com

                                                                                Ever heard of the "death spiral" of health insurance?  Well, here it how it works.  

                                                                                Health insurance claims are dominated by the most expensive participants - see the blue line in Figure 1.  The orange line shows the total collected for the various members if the revenue is 20% over cost.
                                                                                Image:The Death Spiral of health insurance
                                                                                Fig.1. Total cost of health plan members starting from the most costly (blue line).  

                                                                                The costliest 1% represent 40% of the total cost, the costliest 10% represent 80% of the cost.  

                                                                                Income (orange line) crosses the cost curve near the full membership.





                                                                                When is the break-even point?  This is better illustrated in Fig. 2.  The cost function (blue line) crosses the revenue line (orange line) near full membership.  If the revenues are 20% over cost, then the break-even point is at 83% of membership.  If the revenues are X over cost then the break-even point is about 1-X.  

                                                                                Image:The Death Spiral of health insurance
                                                                                Fig.2. Total cost of health plan members starting from the most costly (blue line) and the corresponding income (orange line).  

                                                                                If the revenue is 20% higher than the cost, then the break-even point is roughly at 83%.  

                                                                                If the revenue is 1+X of the cost then the break-even point is at 1/(1+X) which is roughly 1-X for small X.






                                                                                Of course, the break-even point for the policy holders is much lower than that.  As shown in Fig. 3, it is about 13%.  Thus 87% of plan members lose money by being members.  Since health insurance costs are so large, it is tempting to forgo it.  If the 5% healthiest members leave the plan, the insurance company has to increase the price for the remaining members by 5%.  If then the next healthiest 5% balk at the cost increase, the new price goes up another 5% and so forth. At the same time the plan members get sicker with time which also increases costs.

                                                                                That is the "death spiral" of health insurance.  It does not occur for other insurance types, presumably because the associated costs are so much smaller:  insuring one's house is only about $500 per year while health insurance for a family is $20-$30,000 per year.  

                                                                                It would seem that one way to remove the death spiral would be to only insure against catastrophic costs. However, ObamaCare does not allow this for anyone but the poorest members and, presumably, health insurance companies would be shrinking without the gigantic premiums and health care providers would get many fewer customers if regular care would be paid by the patient instead of a third party.
                                                                                Image:The Death Spiral of health insurance
                                                                                Fig.3. Break-even point for members occurs at a low 13th percentile of the costliest members.





                                                                                We at Avalon can assist with your statistics needs.  We use SPSS Modeler, Python and R to get the results.
                                                                                Call us at 646 9825 9080  x1114 to get a quote.
                                                                                Comments Disabled

                                                                                Estimating the cost of your doctor

                                                                                Eugen G Tarnow  October 27 2015 09:35:47 AM
                                                                                By Eugen Tarnow, Ph.D.
                                                                                Avalon Business Systems, Inc.
                                                                                http://AvalonAnalytics.com

                                                                                Doctor charges can vary widely - the same visit can result in different number of charged items and different charges for each item.

                                                                                So we created CostOfDoctors.com,  a website which estimates doctor charges (using our own algorithm).  We do not guarantee results but hope it might be helpful for both those with insurance (some doctors may charge more items per visit) as well as those without insurance.
                                                                                Comments Disabled

                                                                                College Data: An additional $1 tuition gives you $0.45 extra salary six years after college enrollment

                                                                                Eugen G Tarnow  October 6 2015 11:02:53 AM
                                                                                By Eugen Tarnow, Ph.D.
                                                                                Avalon Business Systems, Inc.
                                                                                http://AvalonAnalytics.com

                                                                                From the new college attendance data released by the Obama administration, one finds that earnings six years after enrollment are weakly but positively correlated with the cost of attendance.  For every $1 in extra tuition, the yearly salary increases by $0.45 per year.  

                                                                                Is this enough to justify an expensive school?  If the college education took 4 years, it would take another 9 years to pay off the additional cost.  Is it worth it - you decide!  

                                                                                Image:College Data: An additional $1 tuition gives you $0.45 extra salary six years after college enrollment



                                                                                We at Avalon can assist with your statistics needs.  We use SPSS Modeler, Python and R to get your results.

                                                                                Call us at 646 9825 9080  x1114 to get a quote.

                                                                                Comments Disabled

                                                                                How Avvo games the Google search engine

                                                                                Eugen G Tarnow  October 6 2015 10:42:39 AM
                                                                                By Eugen Tarnow, Ph.D.
                                                                                Avalon Business Systems, Inc.
                                                                                http://AvalonAnalytics.com

                                                                                Those who pay for SEO can get their monies worth.  One company that thrives on using SEO is Avvo.  If you look up your favorite lawyer, chances are that an Avvo "rating" will show up nearby.  This "rating", perhaps without published reviews, will often be close to 6.2, not low enough to get the lawyer upset but not high enough that it can be ignored, perhaps to encourage the lawyer to sign up for the Avvo service.  

                                                                                How is Avvo able to get so close to the real lawyer entries?  If one clicks on the Avvo link one finds 5 mentions of the lawyer name.  If one then looks at the page "source" the lawyer name is mentioned no fewer than 30 times! Google perhaps mistakenly thinks the information is important and there you have it.

                                                                                Avvo has a sense of humor.  They write:

                                                                                "Much as Google doesn’t share its search engine secrets, we don’t share the inner-workings of the Avvo Rating so we can maintain the neutrality and integrity of the rating."


                                                                                Interestingly, lawyer marketing services seem to be closely aligned with Avvo.  There are not too many sites where you see a straightforward assessment of Avvo.  Here is one: http://www.abajournal.com/news/article/another_lawyer_sues_avvo_rating_site_claims_its_practices_are_beyond_unfair/

                                                                                We at Avalon can assist with your statistics needs.  We use SPSS Modeler, Python and R to get your results.

                                                                                Call us at 646 9825 9080  x1114 to get a quote.

                                                                                Comments Disabled

                                                                                College Data: Admitting higher SAT scores yields higher eventual earnings, especially for the SAT math score

                                                                                Eugen G Tarnow  September 22 2015 12:00:17 PM
                                                                                By Eugen Tarnow, Ph.D.
                                                                                Avalon Business Systems, Inc.
                                                                                http://AvalonAnalytics.com

                                                                                One can find lots of interesting things in the new college dataset.  For example, how does the college average SAT score impact eventual earnings?  For every average point on the SAT verbal test, the income increases by $55 per year (see Fig. 1).  100 points, $5,500 per year.  The relationship is even more important for the SAT math score: 100 points, $6,900 per year (see Fig. 2).  So if colleges want better earning alumnae, they should admit higher SAT scoring students.  And the math score is more important than the verbal score.

                                                                                But in general, colleges tend to not care.  In fact, the average SAT math score is skewed towards low scores (see Fig. 3).  Perhaps a current student is more important to the college income than the average alumnus income.

                                                                                Image:College Data: Admitting higher SAT scores yields higher eventual earnings, especially for the SAT math score
                                                                                Fig. 1.  Mean earnings six year after initial enrollment as a function of the average SAT verbal score.


                                                                                Image:College Data: Admitting higher SAT scores yields higher eventual earnings, especially for the SAT math score
                                                                                Fig. 2.  Mean earnings six year after initial enrollment as a function of the average SAT math score.


                                                                                Image:College Data: Admitting higher SAT scores yields higher eventual earnings, especially for the SAT math score
                                                                                Fig. 3.  Distribution of average SAT math scores of admitted students.  The distribution is skewed towards low math scores.



                                                                                We at Avalon can assist with your statistics needs.  We use SPSS Modeler, Python and R to get your results.

                                                                                Call us at 646 9825 9080  x1114 to get a quote.

                                                                                Comments Disabled

                                                                                Fannie Mae mortgages and credit scores

                                                                                Eugen G Tarnow  September 16 2015 10:25:47 AM
                                                                                By Eugen Tarnow, Ph.D.
                                                                                Avalon Business Systems, Inc.
                                                                                http://AvalonAnalytics.com

                                                                                Ever wondered how your credit score affects the interest rate?  Well, in 2013 the interest rate decreased on average 0.2% for every 100 point increase in credit score.  

                                                                                Nevertheless, the credit score only amounted to 7% of the variance.

                                                                                Image:Fannie Mae mortgages and credit scores
                                                                                Fig. 1. Interest as a function of borrower's credit score.


                                                                                Mortgages typically go to partners with very small differences in credit score (see Fig. 2), the standard deviation is only 30 points, and there are almost no mortgages to partners with a credit score difference of 100 or more.  
                                                                                Image:Fannie Mae mortgages and credit scores
                                                                                Fig. 2.  Distribution of differences in credit scores for borrower and co-borrower


                                                                                This suggests four explanations: that we marry those with the same credit score, that lenders do not lend if the scores are too different, that the credit score process artificially forces the scores to be similar, or that the mortgage process tends to force credit scores to become similar over time. The difference in credit scores is somewhat larger for first time home buyers (standard deviation is 40 points instead of 30 points for all mortgages, the overall credit score is 15 points lower per borrower), but that could does not exclude any of the four explanations (we may divorce those with different credit scores, the lenders are encouraged to give loans to first time home buyers, etc.), see Fig. 3.
                                                                                Image:Fannie Mae mortgages and credit scores
                                                                                Fig. 3.  Distribution of differences in credit scores for borrower and co-borrower, first-time home buyers only.


                                                                                And there are very few mortgages in which at least one credit score is 600 or below.


                                                                                Image:Fannie Mae mortgages and credit scores
                                                                                Fig. 4.  Frequency of mortgages as a function of credit score.


                                                                                And one might ask - how did the distribution of credit scores change from 2000 to 2007 to 2013?  In Fig. 5 we see that the distribution became bimodal in 2007!

                                                                                Image:Fannie Mae mortgages and credit scores
                                                                                Image:Fannie Mae mortgages and credit scores
                                                                                Image:Fannie Mae mortgages and credit scores
                                                                                Fig. 5.  Number of mortgages as a function of credit score during the years 2000 (top), 2007 (middle) and 2013 (bottom).  Notice the appearance of two peaks during the mortgage bubble in 2007, almost as if 700 was the new 800.


                                                                                In 2007 the total credit score became the same for first time home buyers as for everyone else. The standard deviation of the difference in credit scored increased from 40 to 50 for first time home buyers and from 30 to 40 for everyone else.  If we divide the borrowers into the two peaks of Fig. 5 (middle panel), the standard deviation of the difference in mortgage score for the top group (borrower credit score above 725) is 35 and for the bottom peak it is 50.  Thus relaxing the borrowing conditions allows the difference in credit scores to increase.

                                                                                We at Avalon can assist with your statistics needs.  We use SPSS Modeler, Python and R to get your results.

                                                                                Call us at 646 9825 9080  x1114 to get a quote.

                                                                                Comments Disabled

                                                                                Think you have a good mortgage rate - think again!

                                                                                Eugen G Tarnow  September 15 2015 04:08:06 PM
                                                                                By Eugen Tarnow, Ph.D.
                                                                                Avalon Business Systems, Inc.
                                                                                http://AvalonAnalytics.com

                                                                                Fannie Mae publishes data on the mortgages they get. There is a delay of more than a year so the latest complete file is from 2013.

                                                                                This is what the distribution of a 1% sample of thirty year mortgage rates looks like:

                                                                                Image:Think you have a good mortgage rate - think again!
                                                                                The most common mortgage rate is 3.68% but there are people who were able to get a 3.06% rate!

                                                                                Think you got a good deal?  Think again! (Unless, of course, the low interest loans had points paid up-front)

                                                                                We at Avalon can assist with your statistics needs.  We use SPSS Modeler, Python and R to get the results.

                                                                                Call us at 646 9825 9080  x1114 to get a quote.

                                                                                Comments Disabled

                                                                                An unpopular drug

                                                                                Eugen Tarnow  September 4 2015 12:39:56 PM
                                                                                By Eugen Tarnow, Ph.D.
                                                                                Avalon Business Systems, Inc.
                                                                                http://AvalonAnalytics.com

                                                                                There are some drugs that are very unpopular with patients. "X" is one of them.  Here are its ratings:

                                                                                Image:An unpopular drug
                                                                                Figure 1.  Number of ratings as a function of rating.


                                                                                One might wonder whether there is anything redeeming about the drug?  For example, can it be prescribed under different circumstances to make it more palatable?

                                                                                Image:An unpopular drug
                                                                                Figure 2. Number of ratings as a function of condition for which it was prescribed.


                                                                                Does not look like it can!  Since 78% of patients rates the drug as a complete flop, there is not even any room left over even for the placebo effect...

                                                                                What is its most common side-effect?  Varieties of pain!

                                                                                Image:An unpopular drug
                                                                                Figure 3.  Words preceding "pain" in patient reviews.



                                                                                We at Avalon can assist with your statistics needs.  We use SPSS Modeler, Python and R to get the results.

                                                                                Call us at 646 9825 9080  x1114 to get a quote.

                                                                                  Changing the survey question, getting a different result but no correction issued by Quinnipiac University

                                                                                  Eugen G Tarnow  August 29 2015 11:17:16 AM
                                                                                  By Eugen Tarnow, Ph.D.
                                                                                  Avalon Business Systems, Inc.
                                                                                  http://AvalonAnalytics.com

                                                                                  We are survey experts and once in a while we notice disturbing survey results.

                                                                                  These are the results in question:

                                                                                  April 27, 2015 - American Voters Back Iran Deal By Wide Margin, Quinnipiac University National Poll Finds


                                                                                  and

                                                                                  August 3, 2015 - American Voters Oppose Iran Deal 2-1, Quinnipiac University National Poll Finds


                                                                                  Really?  How is it possible that American voters changed their minds so quickly and fundamentally?  

                                                                                  It turns out they may not have, but the questions asked were different.  In particular, the question asked in the April poll was a leading question:

                                                                                  "55. As you may know a preliminary agreement was reached in which the United States and other countries would lift major economic sanctions against Iran, in exchange for Iran restricting its nuclear program in a way that makes it harder for it to produce nuclear weapons. Do you support or oppose this agreement?"

                                                                                  The meaning of the Iran deal was laid out tor the voters, instead of letting the voters decide what it meant.

                                                                                  In the August poll the question was neutral:

                                                                                  "67. Do you support or oppose the nuclear deal with Iran?"

                                                                                  Quinnipiac University did not issue a retraction of the earlier survey, nor did they discuss how they changed the question for the August survey.  

                                                                                  A similarly leading question on Iran was used in a poll commissioned from Social Science Research Solutions by the LA Jewish Journal.  They asked:

                                                                                  "As you know, an agreement was reached in which the United States and other countries would lift major economic sanctions against Iran, in exchange for Iran restricting its nuclear program in a way that makes it harder for it to produce nuclear weapons. Do you support or oppose this agreement, or don’t know enough to say?"

                                                                                  They got a result very similar to the April Quinnipiac poll.  There has been no retraction or discussion of the question in the Jewish Journal.

                                                                                  And the bias pattern repeats with the poll of the Cato Institute.  They asked:

                                                                                  "Do you favor or oppose an agreement in which the United States and other countries that would ease oil and economic sanctions on Iran for 10-15 years in return for Iran agreeing to stop its nuclear program over that period?"

                                                                                  Their poll took place July 14-16 and they got the same result as the Qunnipiac April poll.  No correction has been issued in light of the Quinnipiac August poll results.

                                                                                  We at Avalon can assist with your statistics needs.  We use SPSS Modeler, Python and R to get the results.

                                                                                  Call us at 646 9825 9080  x1114 to get a quote.

                                                                                  Comments Disabled

                                                                                  Internet drug ratings: AskAPatient.com & WebMD correlate well but Drugs.com does not

                                                                                  Eugen G Tarnow  July 21 2015 02:42:33 PM
                                                                                  By Eugen Tarnow, Ph.D.
                                                                                  Avalon Business Systems, Inc.
                                                                                  http://AvalonAnalytics.com

                                                                                  Internet drug ratings can help consumers (ever tried reading an FDA approved drug insert?).

                                                                                  Here I compare three internet drug rating sites: AskAPatient.com, Drugs.com and WebMD.com.  For the reasons below, I recommend AskAPatient.com as well as WebMD.com

                                                                                  AskAPatient.com and WebMD.com (satisfaction rating) rates on a scale of 1-5 and Drugs.com on a scale of 1-10 so the latter ratings were scaled appropriately to enable the comparison.  The ratings for ten drugs (the "frequently rated drugs" on AskAPatient.com as of 7/22/15) are shown in Fig. 1.

                                                                                  Image:Internet drug ratings: AskAPatient.com & WebMD correlate well but Drugs.com does not
                                                                                  Fig. 1.  Ratings for 10 drugs.


                                                                                  The results are interesting.   Ratings on AskAPatient.com and WebMD.com are more negative than on Drugs.com (average rating is 2.6 and 2.8 compared to 3.8).  They are also more varied than on Drugs.com.  On AskAPatient.com the ratings vary between 1.6-3.4, on WebMD between 2.0-3.5 and on Drugs.com ratings vary between 3.4 and 4.2.  (the standard deviation for AskAPatient.com is 0.64, for WebMD it is 0.54, while for Drugs.com it is a much smaller 0.29).

                                                                                  AskAPatient.com and WebMD largely agree on their ratings (correlation is 92%) while AskAPatient.com and Drugs.com do not correlate (correlation is only 23%), see Fig. 2.

                                                                                  Image:Internet drug ratings: AskAPatient.com & WebMD correlate well but Drugs.com does not
                                                                                  Fig. 2.  Ratings for 10 drugs show little agreement between AskAPatient.com and Drugs.com


                                                                                  The ratings I compared are shown in Table 1.
                                                                                  AskAPatient.com Drugs.com WebMD.com
                                                                                  Levaquin
                                                                                  1.7
                                                                                  3.4
                                                                                  2.0
                                                                                  Lexapro
                                                                                  3.2
                                                                                  4.2
                                                                                  3.5
                                                                                  Cymbalta
                                                                                  2.8
                                                                                  3.6
                                                                                  3.0
                                                                                  Effexor
                                                                                  3.1
                                                                                  3.9
                                                                                  3.2
                                                                                  Topamax
                                                                                  3
                                                                                  3.7
                                                                                  3.3
                                                                                  Lamictal
                                                                                  3.4
                                                                                  4.2
                                                                                  3.5
                                                                                  Zyrtec
                                                                                  2.1
                                                                                  4.0
                                                                                  2.8
                                                                                  Cipro
                                                                                  1.6
                                                                                  3.9
                                                                                  2.1
                                                                                  Flagyl
                                                                                  2.5
                                                                                  3.7
                                                                                  2.6
                                                                                  Depo-Provera
                                                                                  2.1
                                                                                  3.7
                                                                                  2.4











                                                                                  Table 1.  Ratings for 10 drugs.


                                                                                  We at Avalon can assist with your statistics needs.  We use SPSS Modeler, Python and R to get the results.

                                                                                  Call us at 646 9825 9080  x1114 to get a quote.

                                                                                    Unemployment after 2008 is hidden in a lower labor force participation rate

                                                                                    Eugen G Tarnow  July 1 2015 02:40:39 PM
                                                                                    By Eugen Tarnow, Ph.D.
                                                                                    Avalon Business Systems, Inc.
                                                                                    http://AvalonAnalytics.com

                                                                                    We have all heard that the unemployment has gone back to levels before 2008:

                                                                                    Image:Unemployment after 2008 is hidden in a lower labor force participation rate

                                                                                    You may also have heard about the labor participation rate declining:

                                                                                    Image:Unemployment after 2008 is hidden in a lower labor force participation rate

                                                                                    Well if we combine the two, the effective unemployment rate is:

                                                                                    Image:Unemployment after 2008 is hidden in a lower labor force participation rate

                                                                                    As you can see, it went up after 2008 and reached a plateau in 2010.  It has not gone down since.

                                                                                      Predictive Analytics World Chicago 2015 - savings from predictive analytics

                                                                                      Eugen G Tarnow  July 1 2015 12:04:40 PM
                                                                                      By Eugen Tarnow, Ph.D.
                                                                                      Avalon Business Systems, Inc.
                                                                                      http://AvalonAnalytics.com

                                                                                      Predictive Analytics World in Chicago this year was a very interesting event.  The keynote speakers highlighted two ways to gain from predictive analytics.

                                                                                      Jack Levis, Sr. Director of Process Management at UPS, showed how using predictive analytics to control the packing of the UPS trucks and the delivery paths saved them $300 million per year.  Their yearly revenue is $58 billion yielding a savings from their computer system of 0.5% and their profit is $3 billion meaning that the computer system increased the profit by 10%.  There are 500 employees working on the computer system.  It was not clear whether these salaries were included in the savings.

                                                                                      The other keynote speaker, Rajeeve Kaul, former Senior Vice President of  OfficeMax, did a complex pricing experiment and came up with a better pricing model which saved $20-30 million that year.  Their yearly revenue was $4 billion yielding a savings from their analytics experiment of about 0.5%, just like UPS, and out of a profit of $400 million the analytics experiment was about 5%.

                                                                                      The conclusion seems to be that if given authority over the core business, predictive analytics can save about 0.5% of the revenue and add 5-10% to the profit.

                                                                                      Not bad.

                                                                                      No Correlation Between Executive Pay & Shareholder Return

                                                                                      Eugen G Tarnow  June 24 2015 10:04:13 AM
                                                                                      By Eugen Tarnow, Ph.D.
                                                                                      Avalon Business Systems, Inc.
                                                                                      http://AvalonAnalytics.com

                                                                                      Today's Wall Street Journal covers executive pay.  The journalists forgot the "line".  

                                                                                      We generated it and it appears in the graph below of executive pay versus shareholder return.  The line is dotted and the number we were all looking for is indicated above.  R2=0.0001.

                                                                                      In other words, there is no correlation between executive pay and shareholder return.

                                                                                      Image:No Correlation Between Executive Pay & Shareholder Return

                                                                                      Was the show "House, MD" correlated with vicodin use? If so, which way did the causation go?

                                                                                      Eugen G Tarnow  May 13 2015 10:03:37 AM
                                                                                      By Eugen Tarnow, Ph.D.
                                                                                      Avalon Business Systems, Inc.
                                                                                      http://AvalonAnalytics.com

                                                                                      A very famous social psychologist, Stanley Milgram, once tried to find out whether violence on TV translates into violence in society.  His findings were that the crime rate did not go up after a crime was shown on TV, BUT that the type of crime tended to imitate the TV show.

                                                                                      When you do predictive analytics, all kinds of ideas enter your mind when you sit and clean client data.

                                                                                      Recently it was found that the number one drug for Medicare patients is vicodin, the drug that gets Dr. House hooked on TV.  The question I ask myself is - did the TV show influence the number of prescriptions of vicodin?  If so, in which direction?

                                                                                      We don't have the resources to do this project, but someone ought to do it.  Help has been offered on youtube: https://www.youtube.com/watch?v=PSaasBjeJek

                                                                                      One would need to get the daily prescription data from IMS and then the show data (from Nielsen?).  Entirely doable and fascinating project!

                                                                                      Having a heart attack? Medicare total payments to a hospital does not guarantee a better outcome.

                                                                                      Eugen G Tarnow  May 11 2015 09:40:58 AM
                                                                                      By Eugen Tarnow, Ph.D.
                                                                                      Avalon Business Systems, Inc.
                                                                                      http://AvalonAnalytics.com

                                                                                      Thanks to a FOIA by the Wall Street Journal, Medicare now releases datasets that describe what the tax payer gets for the yearly 0.5 trillion dollar payments.  

                                                                                      For example, do we pay more for better quality treatments?  When it comes to heart attacks the data shows that Medicare payments hardly correlate with quality.  Below I show you the hospital total charges related to heart attack care and the corresponding "score" (% complications - lower is better).  Medicare payments represents only about 7% of the variance and even then you need a $10 million payment to lower the rate of complications from say 15% to 14%.

                                                                                      Image:Having a heart attack?  Medicare total payments to a hospital does not guarantee a better outcome.

                                                                                      Do hospitals that charge more per heart attack do a better job?  Here the correlation is even smaller - only 1.5% of the score is associated with price.  Medicare should probably cut the payments which range from $17,000 to $27,000 to the bottom of the range.  

                                                                                      Image:Having a heart attack?  Medicare total payments to a hospital does not guarantee a better outcome.
                                                                                      In other words, if you are having a heart attack, you might as well save the money in your wallet and go to a hospital that charges Medicare (and you) less for the treatment.

                                                                                      IBM Vision - Two talks on predictive analytics

                                                                                      Eugen G Tarnow  May 5 2015 01:44:28 PM
                                                                                      By Eugen Tarnow, Ph.D.
                                                                                      Avalon Business Systems, Inc.
                                                                                      http://AvalonAnalytics.com

                                                                                      At this month's Vision event in Orlando, there will be at least two talks about the power of SPSS Modeler.  

                                                                                      One will be about predictive modelling at Charles Schwab: validating the "drivers" and using them for better forecasting.  There is no information yet as to what is being forecast.  The stock market?  The demand from customers?

                                                                                      The other will be about Huffy bicycles: optimizing inventory forecasting and production schedules.

                                                                                      Predictive analytics software used to size up patent targets, preparing a hedge fund for attack

                                                                                      Eugen G Tarnow  April 8 2015 08:31:03 AM
                                                                                      By Eugen Tarnow, Ph.D.
                                                                                      Avalon Business Systems, Inc.
                                                                                      http://AvalonAnalytics.com

                                                                                      In today's Wall Street Journal, http://www.wsj.com/articles/hedge-fund-manager-kyle-bass-challenges-jazz-pharmaceuticals-patent-1428417408, a hedge fund is going after pharmaceutical patents and, at the same time, shorting their stock.  The idea is to use a legal channel, "Inter Partes Review", invented to invalidate patents of patent trolls, against pharmaceutical companies.

                                                                                      How do they find the pharmaceutical companies they think are vulnerable?  Apparently "Mr. Spangenberg [consultant to the hedge fund] sizes up potential patent targets, using a predictive analytics software program he acquired several years ago."

                                                                                      There is no description of the analytics software but here is a guess.  According to the article, the Inter Partes Review process invalidates about 77% of the patents before it.  The software, with the help of a patent lawyer, searches all those decisions for commonalities.  An analytics model is trained on those commonalities to see if it would pick out the correct 77% of invalidated patents.  Once the model is optimized it is then applied to the full patent database (which can easily be downloaded). The model then predicts which patents are most likely to be invalidated.  The hedge fund leadership then selects which companies are most vulnerable to the predicted patent failures and attacks those.

                                                                                      A new business model in the making, created on analytics.  We can do it!

                                                                                      The
                                                                                      Avalon Predictive Analytics team can assist with your analytics needs.  

                                                                                      Medicare pricing distortion raises healthcare costs by 45%

                                                                                      Eugen G Tarnow  June 12 2014 04:15:22 PM
                                                                                      By Eugen Tarnow, Ph.D.
                                                                                      Avalon Business Systems, Inc.
                                                                                      http://AvalonAnalytics.com

                                                                                      As ObamaCare has been implemented we are finding out that even the subsidized rates are too high for many people and that the rates will keep going up.  One of the reasons for this may have nothing to do with ObamaCare but everything to do with Medicare.

                                                                                      Medicare recently released a dataset, prompted by a freedom of information request from the Wall Street Journal, which includes most approved claims for CY2012.  My analysis of this dataset shows that Medicare causes significant price distortions.

                                                                                      A price distribution should follow what is called a log-normal distribution (like housing prices in the UK, for example) and look like this (below).  To be as inclusive as possible, I have plotted the probability of charging a certain list price as a multiple of the Medicare largest reimbursed price – that way I can average over all Medicare procedures.
                                                                                      Image:Medicare pricing distortion raises healthcare costs by 45%

                                                                                      Contrast this theoretical curve with what the provider pricing actually looks like for Medicare data from 2012.  I have limited it to procedures that Medicare reimburses more than $100 per encounter:

                                                                                      Image:Medicare pricing distortion raises healthcare costs by 45%

                                                                                      As you can see, the left tail of the distribution is gone.  All providers, without exceptions, charge at least the Medicare largest reimbursement.  Medicare provides a price floor.  It is a customer that is so huge (14% of the population with the majority of the nation’s health problems and the time to take care of them), that a provider (91% of non-pediatric physicians accept new Medicare patients, almost all have existing Medicare patients) can always find a person covered by Medicare and would never bother with any patient who cannot pay the Medicare price.

                                                                                      The distortion caused by the Medicare price floor increases list prices.  I can estimate just how much by fitting a log-normal distribution to the actual price distribution and I get this:

                                                                                      Image:Medicare pricing distortion raises healthcare costs by 45%

                                                                                      From the fit I find that if there was no Medicare price floor, the log-normal pricing curve would yield an average list price of 2.05 * Medicare.  Once Medicare comes in, all the providers raise their list prices above the Medicare price floor.  The list prices that change the most are the lowest ones.  And some providers make this particularly obvious; they simply charge multiples of Medicare (peaks in pricing at 2, and 3 times Medicare).

                                                                                      So how much does Medicare increase the list prices?  The list price after the Medicare price floor is 2.99 * Medicare, an increase of 45% from the log-normal distribution!

                                                                                      In NJ we have a little publicized law which states that providers can only charge 20% over Medicare for patients making less than 4 times the poverty limit.  The effect of this law is to provide yet another incentive for providers to increase their prices.

                                                                                      What could the government do to curb ever rising health care costs (without the corresponding rise in quality)?

                                                                                      1. Medicare rates could be set much lower to minimize distorting health care prices for everyone else.  Since current Medicare recipients only paid for 35% of its cost this would seem fair.

                                                                                      2. Laws that say that a provider can only get the maximum reimbursement from Medicare if their list price is higher than that of Medicare have to be repealed.  This may be a form of price fixing.

                                                                                      3.  Medicare could be thought of as a privilege and this privilege could come with a hook: if you want Medicare patients, you must also treat non-Medicare patients for the same price.

                                                                                      The
                                                                                      Avalon Predictive Analytics team can assist with your analytics needs.  

                                                                                        IBM Big Data is growing at 14% a year

                                                                                        Eugen G Tarnow  June 12 2014 04:14:05 PM
                                                                                        By Eugen Tarnow, Ph.D.
                                                                                        Avalon Business Systems, Inc.
                                                                                        http://AvalonAnalytics.com

                                                                                        I recently went to a Big Data conference hosted by IBM and things look good:  IBM's Big Data division is growing at 14% per year!

                                                                                        The
                                                                                        Avalon Predictive Analytics team can assist with your analytics needs.