Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Any active users here? #302

Open
derek-adair opened this issue Aug 13, 2018 · 36 comments
Open

Any active users here? #302

derek-adair opened this issue Aug 13, 2018 · 36 comments

Comments

@derek-adair
Copy link

In light of our great leader @BurntSushi leaving the project, I've forked the nflgame repo.

Is anyone using this project actively?

@g24swint
Copy link

I forked nfldb with the idea that I might a) get it off of deprecated python, and then b) play with it eventually.

But I would not describe myself as terribly active with it. I'm not really using nflgame repo at all, yet. It has the same HUGE problem of being only deprecated Python, also

@JoeNoPhoto
Copy link

Actively trying to use it. Just found your nflgame fork and now I need to figure out where the most active fork of nfldb is. I'm in the process of trying to figure out how to port this all over to Ruby instead.

@ochawkeye
Copy link
Contributor

ochawkeye commented Aug 21, 2018

I still use nfldb.

S:\>python
Python 2.7.14 (v2.7.14:84471935ed, Sep 16 2017, 20:19:30) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import nfldb
>>> db = nfldb.connect()
>>> q = nfldb.Query(db)
>>> q.game(season_year=2018, week=2, season_type='Preseason')
<nfldb.query.Query object at 0x027E63F0>
>>> for game in q.as_games():
...     print game
...
Preseason 2018 week 2 on 08/20 at 07:00PM, BAL (17) at IND (10)
Preseason 2018 week 2 on 08/18 at 07:00PM, TB (30) at TEN (14)
Preseason 2018 week 2 on 08/18 at 12:00PM, JAC (14) at MIN (10)
Preseason 2018 week 2 on 08/18 at 03:00PM, OAK (15) at LA (19)
Preseason 2018 week 2 on 08/18 at 09:00PM, SEA (14) at UNK (24)
Preseason 2018 week 2 on 08/18 at 07:00PM, SF (13) at HOU (16)
Preseason 2018 week 2 on 08/18 at 08:05PM, CHI (24) at DEN (23)
Preseason 2018 week 2 on 08/18 at 06:00PM, CIN (21) at DAL (13)
Preseason 2018 week 2 on 08/17 at 07:00PM, ARI (20) at NO (15)
Preseason 2018 week 2 on 08/17 at 06:00PM, NYG (30) at DET (17)
Preseason 2018 week 2 on 08/17 at 06:30PM, BUF (19) at CLE (17)
Preseason 2018 week 2 on 08/17 at 06:30PM, MIA (20) at CAR (27)
Preseason 2018 week 2 on 08/17 at 06:00PM, KC (28) at ATL (14)
Preseason 2018 week 2 on 08/16 at 06:30PM, PHI (20) at NE (37)
Preseason 2018 week 2 on 08/16 at 07:00PM, PIT (34) at GB (51)
Preseason 2018 week 2 on 08/16 at 07:00PM, NYJ (13) at WAS (15)
>>>

@derek-adair
Copy link
Author

actively trying to port to ruby

Oy vey... Good luck! I wouldn't recommend that as @ochawkeye as demonstrated, this project is in working order, albiet a bit quirky. I'm in the middle of prepping for this season so i can't really take on any maintenance questions or the like.

I'm ASSUMING as a new user you are unable to access new seasons data?

@weissleb
Copy link

weissleb commented Aug 26, 2018

I'm attempting to use this, but cannot pull updates. See my issue.

UPDATE: I resolved this issue by simply downloading the source code and extracting the scripts folder. Additionally, I had to do all the other stuff folks have been discussing, including adding JAX and LAC to the teams (#289 , this solution) , and installing nflgame directly from the git repo (#296 , this solution).

All seems in order now.

@JoeNoPhoto
Copy link

@derek-adair, I've been fiddling with nfldb off and on for the last year or so. I've got the new season's data working fine(in python). I haven't gotten to the point of updating data in Rails. I'm still trying to port the postgres db into Rails which is a PITA in itself.

@derek-adair
Copy link
Author

derek-adair commented Sep 5, 2018

@weissleb you may try our fork.

@JoeNoPhoto
Copy link

@derek-adair edit your link to remove '-redux' from the end. The package is redux but it's still a fork of nflgame.

@derek-adair
Copy link
Author

Ya i mean i'm unsure of the subtleties of naming a project "redux" it's just something i saw some contributors do. It's a temporary repository so the name doesn't really matter.

@ochawkeye
Copy link
Contributor

ochawkeye commented Sep 6, 2018

I think he means that the link you pasted to your fork in #302 (comment)
doesn't actually go to your fork. It's a 404 error.

@derek-adair
Copy link
Author

hah. Fixed.

@derek-adair
Copy link
Author

damn i keep hitting the close button grr.

@MargareeMan
Copy link

Yes, I have returned to update my PostgreSQL nfldb and use nflgame. @ochawkeye and others have provided excellent direction and your (@derek-adair) fork and issue tracking is also helpful. I find @BurntSushi nfldb implementation in PostgreSQL is awesome. His use of custom data types and domains, triggers and procedures is well beyond my abilities; but an absolute joy to use and try to reverse engineer. His ER diagrams are very useful and can assist in building your own db.

I think the only stumbling block to using nfldb as is (from the 2016 code) is to account for the JAC-JAX issue, the LA Rams and the LAC Chargers. That is straight forward INSERT SQL. As well, as the various threads indicate it is necessary to alter nflgame_init_.py and nfldb\team.py (both team1 and team2) code.

Also, as @ochawkeye points out, make sure you have a clean install with the required packages.

@jaredkwright
Copy link

Is anyone working on a fork of nfldb that is using nflgame-redux from @derek-adair ?

@ochawkeye
Copy link
Contributor

'nfldb' works with @derek-adair's 'nflgame-redux'.

@jaredkwright
Copy link

'nfldb' works with @derek-adair's 'nflgame-redux'.

Oh yeah I meant basically someone releasing a nfldb-redux or something that would be installable from a package manager and have all the updates needed to hit the ground running.

@derek-adair
Copy link
Author

I plan on doing this when i get some extra time

Hopefully soon

@MargareeMan
Copy link

MargareeMan commented Dec 6, 2018

@derek-adair , @ochawkeye @jaredkwright ,

I think @BurntSushi nfldb (0.2.17) is a "hit the ground running" package. Only two lines of code need to be added to the team.py file and you are good to go. All the challenges of getting a fully functional PostgreSQL nfldb database installed and working with historical integrity stem from the confusion generated for the use of multiple forks of the nflgame library and the confusing/contradictory issues arising from different users using slightly different configurations of nflgame. In particular, incomplete schedule.json files, outdated/incomplete json.gz files in the gamecenter-json directory, and a code deficiency in update_players.py have caused multiple, intersecting problems. The direct result is that the issues documentation has become more of a hindrance than a help in getting a proper install of the database.

I work on Windows using PyCharm and DataGrip (or DBVisualizer). I use PyCharm to create a Python 2.7 conda env and install the BurntSushi nfldb (0.2.17) and @derek-adair nflgame-redux (1.2.23) libraries into that project.

I use "nickfrerichs/fflproject#29" suggestion and update_players.py file by adding the following two statements after line175:

if name.count(',') == 2:
name = name.strip(', Jr.')

There are other proposed solutions (BurntSushi/nflgame#316).

I verify that the schedule.json file is complete. (I have seem versions missing one game, (i.e., 2017111911, the postponed TB@MIA game. In which case you can manually insert

[
   "2017111911",
   {
    "away": "TB",
    "day": 19,
    "eid": "2017111911",
    "gamekey": "57241",
    "home": "MIA",
    "meridiem": "PM",
    "month": 11,
    "season_type": "REG",
    "time": "1:00",
    "wday": "Sun",
    "week": 11,
    "year": 2017
   }
  ],

Or, I have seen schedule.json files that have 2018 entries but are missing over 100 games from 2017. (In this case, the nflgame update-schedule script never correctly updates the schedule.json file.) The remedy is to look around at the various forks of nflgame until you find a complete file. The crucial link here is that nfldb uses the nflgame schedule.json file to load/update its tables. So if the schedule.json file is incomplete, the database will be incomplete.

Then, if you want a historically complete (from 2009) nfldb you MUST avoid two standard practices:

  1. Do NOT use any of the nfldb.sql.zip files in the various repos (even @BurntSushi), because they have incomplete game drive (hence play, play_players, and agg_play) info for gsis_id 2014092500 (missing drive 26), gsis_id 2014122809' (missing drives 6 to 23 ) and gsis_id 2013111709 (missing drive 27).

  2. Do NOT run the nfldb-update script on a nflgame install where the gamecenter-json .json.gz files have not been refreshed since the deprecation of the @BurntSushi library.

So, finally, assuming you use @derek-adair nflgame-redux (1.2.23) and you have corrected the update_players and schedule.json files, then you are ready to install nfldb (0.2.17).

  1. First run the nflgame update-schedule script, simply to confirm that it runs without error. As of 2018-12-04 it should have 3309 entries, including games yet to be played in 2018 (and including 2017111911). If this does not run correctly, your issue is with nflgame and you are not ready to run nfldb-update.

  2. Now run the nflgame update-players script. It will run (as of 2018-12-04) with one "error" of no consequence to the integrity of nflgame or nfldb. (This is a well known issue with NFL not providing a gsis_id for a player with no stats).

Loading games for REG 2018 week 14
Downloading team rosters...
33/33 complete. (100.00%)
Done!
Fetching GSIS identifiers for players not in nflgame...
8/8 complete. (100.00%)
Done!


There were some errors during the download. Usually this is a
result of an HTTP request timing out, which means the
resulting "players.json" file is probably missing some data.
An appropriate solution is to re-run the script until there
are no more errors (or when the errors are problems on
NFL.com side.)
-------------------------------------------------------------------------------
Could not get GSIS id at http:https://www.nfl.com/player/cedwilson/2560936/profile
-------------------------------------------------------------------------------
Could not find gsis_id for {'status': u'RES', 'last_name': u'Wilson', 'weight': 197, 'profile_id': 2560936, 'number': 16, 'height': 74, 'college': u'Bowie State', 'full_name': u'Ced Wilson', 'first_name': u'Ced', 'years_pro': 0, 'birthdate': u'3/10/1995', 'team': 'DAL', 'position': u'WR', 'profile_url': 'http:https://www.nfl.com/player/cedwilson/2560936/profile'}

You may get additional errors of the type below. However, again they do not affect the integrity of nfldb or nflgame and they will be resolved as the NFL updates their data in gamecenter and as you run future updates.

-------------------------------------------------------------------------------
Could not get profile URL for (00-0033258, D.Daniels)
-------------------------------------------------------------------------------
Could not get profile URL for (00-0034526, C.Reavis)
-------------------------------------------------------------------------------
Could not get profile URL for (00-0033471, I.Ford)
-------------------------------------------------------------------------------

Now that nflgame is working with no errors and full integrity of the schedule.json file, you are one step away from installing nfldb. You MUST first ensure that the files in the gamecenter-json directory have a one-to-one correspondence to the files listed in schedule.json (with the exception of 2017083158 (never played) and 2017091006 (posponed and played as 2017111911) and the yet-to-be played games following the current date. As of 2018-12-04, there are 3309 entries in a current schedule.json file. There are 3245 games that the schedule considers "played". There are 2343 files that are in the gamecenter-json directory). Either you need a source for a correct list of these 2343 files, or you need to regenerate them as you run nfldb-update.)

  1. If you already have an install of nfldb; DROP that database using pgAdmin 4.3 or later.

  2. If you are installing for the first time, use pgAdmin to create a new user (nfldb) with the password (nfldb). (If you intend to follow https://github.com/BurntSushi/nfldb/wiki/Windows-Install, you really only need to do steps 4 to 6. You no longer need step 7 (enable fuzzystrmatch) because of the use of gsis identifiers for all players. You must NOT follow steps 8 thru 10 (preloading the database with anyones nfldb.zip data because there are some missing drive records in these files.) You must follow steps 11 and 12 and create you config.ini file so that you can run the updates and queries).

  3. With pgAdmin, create a new nfldb with the owner nfldb. (When you DROP the database, the nfldb user and credentials will NOT have been dropped. Or you will have created the user in step 2 above).

  4. IF (the big IF) you are certain of the integrity and completeness of the .gz files in your gamecenter-json directory you are now ready to make a two line change to the team.py file in the @BurntSushi nfldb (0.2.17) package and then create (update) your PostgreSQL nfldb database.

  5. If you are not certain of the integrity of both your gamecenter-json directory and the files within; then delete all the json.gz files. The nfldb update process will download all these files from NFL gamecenter (based on your complete schedule.json file). This update/download is a one time event. However, you MUST run this when no games are in progress - preferably on a Tuesday or Wednesday night. Hopefully you can avoid this by having a complete zip file of this directory.

  6. For completeness the team.py dictionary is:

teams1 = [
    ['ARI', 'Arizona', 'Cardinals', 'Arizona Cardinals'],
    ['ATL', 'Atlanta', 'Falcons', 'Atlanta Falcons'],
    ['BAL', 'Baltimore', 'Ravens', 'Baltimore Ravens'],
    ['BUF', 'Buffalo', 'Bills', 'Buffalo Bills'],
    ['CAR', 'Carolina', 'Panthers', 'Carolina Panthers'],
    ['CHI', 'Chicago', 'Bears', 'Chicago Bears'],
    ['CIN', 'Cincinnati', 'Bengals', 'Cincinnati Bengals'],
    ['CLE', 'Cleveland', 'Browns', 'Cleveland Browns'],
    ['DAL', 'Dallas', 'Cowboys', 'Dallas Cowboys'],
    ['DEN', 'Denver', 'Broncos', 'Denver Broncos'],
    ['DET', 'Detroit', 'Lions', 'Detroit Lions'],
    ['GB', 'Green Bay', 'Packers', 'Green Bay Packers', 'G.B.', 'GNB'],
    ['HOU', 'Houston', 'Texans', 'Houston Texans'],
    ['IND', 'Indianapolis', 'Colts', 'Indianapolis Colts'],
    ['JAC', 'Jacksonville', 'Jaguars', 'Jacksonville Jaguars', 'JAX'],
    ['KC', 'Kansas City', 'Chiefs', 'Kansas City Chiefs', 'K.C.', 'KAN'],
    ['MIA', 'Miami', 'Dolphins', 'Miami Dolphins'],
    ['MIN', 'Minnesota', 'Vikings', 'Minnesota Vikings'],
    ['NE', 'New England', 'Patriots', 'New England Patriots', 'N.E.', 'NWE'],
    ['NO', 'New Orleans', 'Saints', 'New Orleans Saints', 'N.O.', 'NOR'],
    ['NYG', 'New York', 'Giants', 'New York Giants', 'N.Y.G.'],
    ['NYJ', 'New York', 'Jets', 'New York Jets', 'N.Y.J.'],
    ['OAK', 'Oakland', 'Raiders', 'Oakland Raiders'],
    ['PHI', 'Philadelphia', 'Eagles', 'Philadelphia Eagles'],
    ['PIT', 'Pittsburgh', 'Steelers', 'Pittsburgh Steelers'],
    ['SD', 'San Diego', 'Chargers', 'San Diego Chargers', 'S.D.', 'SDG'],
    ['SEA', 'Seattle', 'Seahawks', 'Seattle Seahawks'],
    ['SF', 'San Francisco', '49ers', 'San Francisco 49ers', 'S.F.', 'SFO'],
    ['STL', 'St. Louis', 'Rams', 'St. Louis Rams', 'S.T.L.'],
    ['TB', 'Tampa Bay', 'Buccaneers', 'Tampa Bay Buccaneers', 'T.B.', 'TAM'],
    ['TEN', 'Tennessee', 'Titans', 'Tennessee Titans'],
    ['WAS', 'Washington', 'Redskins', 'Washington Redskins', 'WSH'],
    ['UNK', 'UNK', 'UNK'],
]

teams2 = [
    ['JAX', 'Jacksonville', 'Jaguars', 'Jacksonville Jaguars', 'JAC'],
    ['LA', 'Los Angeles', 'Rams', 'Los Angeles Rams', 'L.A.'],
    ['LAC', 'Los Angeles C', 'Chargers', 'Los Angeles Chargers', 'L.A.C'],
]

The only two lines you are changing are the JAX and LAC lines in teams2. That is all. That is the only change to @BurntSushi nfldb (0.2.17). That is why this is a 'hit the road running' package.

When you run nfldb-update script for the first time on an empty PostgreSQL nfldb, it creates a new (version 8) schema (refer to the meta table after your install) and all the tables, indexex, triggers, domains, the agg_play materialized view, etc.

The update takes a while. If you had to empty your gamecenter-json directory to ensure integrity, then the necessary _gtd.json files will, in accordance with schedule.json file contents, be downloaded from "http:https://www.nfl.com/liveupdate/game-center" and then batch loaded into the database. If your .gz files are okay (or you do not need a complete nfldb database), then the upgrade will load whatever is in your gamecenter-json directory. If your schedule.json file is missing a file, but it exists in your gamecenter directory; it will NOT be loaded. The schedule.json file drives the process, but the absence of files in the gamecenter directory can affect the outcome.

A completer process results in output (truncated in the middle section) as follows:

-------------------------------------------------------------------------------
STARTING NFLDB UPDATE AT 2018-12-04 16:19:35.662000
Connecting to nfldb...
MIGRATING DATABASE... PLEASE WAIT

THIS WILL ONLY HAPPEN ONCE.

This is currently adding a play aggregation table (a materialized view) derived
from the `play` and `play_player` tables. Depending on your machine, this
should take less than two minutes (this includes aggregating the data and
adding indexes).

This aggregation table will automatically update itself when data is added or
changed.

Aggregation complete. Adding indexes...
Indexing complete. Adding triggers...
done.
Setting timezone to UTC... done.
Updating player JSON database... (last update was 0001-01-01 00:00:00+00:00)
Loading games for REG 2018 week 13
Downloading team rosters...
33/33 complete. (100.00%)
Done!
Fetching GSIS identifiers for players not in nflgame...
1/1 complete. (100.00%)
Done!


There were some errors during the download. Usually this is a
result of an HTTP request timing out, which means the
resulting "players.json" file is probably missing some data.
An appropriate solution is to re-run the script until there
are no more errors (or when the errors are problems on
NFL.com side.)
-------------------------------------------------------------------------------
Could not get GSIS id at http:https://www.nfl.com/player/cedwilson/2560936/profile
-------------------------------------------------------------------------------
Could not find gsis_id for {'status': u'RES', 'last_name': u'Wilson', 'weight': 197, 'profile_id': 2560936, 'number': 16, 'height': 74, 'college': u'Bowie State', 'full_name': u'Ced Wilson', 'first_name': u'Ced', 'years_pro': 0, 'birthdate': u'3/10/1995', 'team': 'DAL', 'position': u'WR', 'profile_url': 'http:https://www.nfl.com/player/cedwilson/2560936/profile'}
done.
Locking player table...
Updating 7520 players... done.
Locking write access to tables... done.
Updating season phase, year and week... done.
Adding schedule data for 3309 games... done.
Bulk inserting data for 3245 games...
        Sending batch of data to database.
        Sending batch of data to database.
        Sending batch of data to database.
        Sending batch of data to database.
        Sending batch of data to database.
        ....
        Sending batch of data to database.
        Sending batch of data to database.
done.
Updating schedule JSON database...
Last updated: 2018-12-04 16:23:42.871000
done.
Updating schedule for (Regular, 2018, 13)
done.
Closing database connection... done.
FINISHED NFLDB UPDATE AT 2018-12-04 17:09:14.053000
-------------------------------------------------------------------------------

C:\Users\whale\Anaconda2\envs\NFL-Game-Conda>

You now have a clean, hit the road running nfldb.

@jaredkwright
Copy link

@MargareeMan Thank you for this great write-up!

@derek-adair
Copy link
Author

@MargareeMan - let me just say.... wow. one of the most helpful comments i've read in a very long time. Thank you. It belongs in a wiki somewhere and illustrates perfectly how one may leverage these projects.

Do you have any suggestions on how these problems can be fixed? It seems like fork and;

  1. Remove the nfldb.sql.zip from the repo
  2. replace nflgame w/ nflgame-redux

Also, Please note that I intend on removing the dependency on http:https://www.nfl.com/liveupdate/game-center - There is a much more complete stream of data, that seems to be free (just not advertised). This may or may not impact nfldb and this very issue.

@MargareeMan
Copy link

MargareeMan commented Dec 15, 2018

@derek-adair Thank you for your very generous compliment. But, if I have added to this project, it is only because of the efforts of more competent contributors such as yourself and @ochawkeye (and, of course, the solid foundation that inherited from @BurntSushi ).

I cannot advise you on how to proceed with updating the current marriage between nfldb-nflgame. And, a marriage it is. The vows were last renewed on September 8th, 2016. Since then, there have been some small ups and down; but they are still together. It is, frankly, a good marriage based on a strong and sophisticated foundation. Nonetheless, it is fragile. And, although the architecture invites some flexibility, the faint of heart need not apply.

I think subsequent development direction really depends on the audience, the functionally desired, and the "legal" stance the developer takes toward the source(s) of data consumed. I am not a Python programmer (in fact I am not a programmer at all). And, I have no interest in Fantasy football. I am a simple hobbyist that is thankful that the NFL passively "permits" hobbyists to acquire data through their public-facing endpoints. Personally, I consider Andrew Gallant's nfldb/nflgame impressive (functioning) implementation as a good model architecture for linking a SQL database to data feeds.

It remains tied to NFL proprietary data that we, as hobbyists, are "permitted" to use, individually, at the pleasure of the NFL. Therefore, any alteration you want to make to the fork, I think, needs to carefully consider what -- if any -- data you provide along with the application code.

@derek-adair
Copy link
Author

wow...so you are just really good at installing software???

@cmjarrett
Copy link

@MargareeMan Thanks for that magnificent writeup, alongside @derek-adair and @ochawkeye for their contributions. Hoping to help once I get more familiar with the codebase and python--I've got a project I want to finish by labor day and hoping to continue on it during the season using nflgame/db.

@Dubl
Copy link

Dubl commented Sep 18, 2019

hello all,
we have made an Azure version of nfldb. 3 lines of code gets you an updating database on SQL Server.. for a few dollars a month

https://github.com/ml-systems/nfldb

let us know what you think

@derek-adair
Copy link
Author

@Dubl - how disappointing to not open source your work.

@BurntSushi
Copy link
Owner

They did. It's MIT licensed.

@Dubl
Copy link

Dubl commented Sep 18, 2019

@derek-adair Yes it’s open source. All the code is there, it just utilizes Azure resources (storage, db, etc) which costs a nominal amount to run (after the free trial)

If you wanted to, you could just run the python script on your own machine, to pull from the api and convert to csv, and then load that to your local db. We just wrapped it up so a newcomer doesn’t need to learn and install python and postgres. Just run a few lines of code, download a free copy of ssms, and start querying your db.

We’ll do a more thorough writeup on how it works.

@derek-adair
Copy link
Author

Whoops I misread the project's intent. It looked like a pre-baked image that you were charging for.

Very cool.

@derek-adair
Copy link
Author

I need glasses D: lol

@Dubl
Copy link

Dubl commented Sep 20, 2019

“Is the az group create line in DeployNFLDB.ps1 creating a new Resource group? Surely not.”
In fact, yes, that’s exactly what it’s doing!
Amazingly the code is able to create:
-A resource group to house all your resources in
-all your resources, which include a sql server instance and db, and file storage, and connections, and a “function app” which is just a way to bolt on some arbitrary python in the middle of it.
-“logic apps” which are little workflows to run processes (like, query the internet, take the json you get, and run it through a python function, and take that csv output and put it into a db) (this is exactly what the ‘gamedaydata_pfunction_schedule’ app does.

And yes ‘dbadmin’ is a hardcoded db user name.

So... if you are tired of it, or want to start over, just delete the resource group from the portal, and you’re free to start over or to be done.

The ‘functionapp’ argument is just a name to give the function app. (The bolt-in python).

Thanks so much for the questions. Happy to explain/write up more

@BurntSushi
Copy link
Owner

BurntSushi commented Sep 20, 2019

Folks, this is the nfldb issue tracker. I don't have a problem with dropping a link to a project, but please have a bit of courtesy and take discussions of other projects to their respective issue trackers.

@MargareeMan
Copy link

MargareeMan commented Sep 20, 2019

Sorry @BurntSushi . My last post was coincidental with your admonishment. I also apologize for mixing the Azure discussion into this thread. My bad. I have deleted my comments.

@BurntSushi
Copy link
Owner

Can you enlighten me a little on the relationships among the MIT license, the use of the NFL json api and the storage and use of the data retrieved from the api?

No. I'm not a lawyer. So you should go talk to one.

This topic has been discussed many times. Please consider searching the nflgame and nfldb issue trackers.

Failing all of that, use common sense.

@msmorey
Copy link

msmorey commented Dec 17, 2019

I'm pretty green when it comes to complex installations, but I followed @MargareeMan's instructions on Mac OS 10.14.6 Mojave and I was able to get everything working in a couple hours. It's so cool! I did have to overcome a few issues installing psycopg2 to a conda environment in python 2.7, but I was able to get it up and running with a few stack overflow hints. https://stackoverflow.com/questions/16407995/psycopg2-image-not-found

Once I was able to run the update script, everything worked perfect. It took about 2 hours to fully update. Thanks so much for all the help in this thread! I'm off to start exploring stats.

@MargareeMan
Copy link

MargareeMan commented Dec 18, 2019

Congrats, @msmorey that's better than I did on the Mac. I do not know your competence with SQL; but I struggle with JOINS. So I hope I do not embarrass myself (or you) by passing along a query I did last week while watching the BAL-NYJ

SELECT g.week AS "wk",
pg.time,
pg.description,
pp.receiving_rec AS "rec",
pp.receiving_yds AS "yds",
pp.receiving_yac_yds AS "yac",
pp.receiving_tds AS "tds",
p.full_name AS "name",
pp.gsis_id AS "gid",
pp.play_id AS "pid"
FROM play pg
INNER JOIN game g ON pg.gsis_id = g.gsis_id
INNER JOIN play_player pp ON pg.gsis_id = pp.gsis_id AND pg.play_id = pp.play_id
INNER JOIN player p on pp.player_id = p.player_id
WHERE g.season_type = 'Regular'
AND g.season_year = 2019
AND pp.player_id = (
SELECT player_id
FROM player
WHERE team = 'BAL' AND last_name LIKE 'Brown' AND position = 'WR'
)
AND pp.receiving_rec = 1
AND pp.receiving_tds = 1
AND pg.description LIKE '%deep%'
ORDER BY g.week DESC;
I know this pales in comparison ti what @BurntSushi does; but I put it here in case the multiple JOIN is a useful example.

@derek-adair
Copy link
Author

I've forked nfldb and begun a python3 conversion. Currently digesting the entire project, but running python 2to3 and swapping in nflgame-redux>2.0.2 has it at least running.

Any help would be greatly appreciated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests