Research Data gaps + big projects

Remove this Banner Ad

RedmanWasHere said:
1927-41 now done.
Click to expand...


Been a while between updates but 1945-60 are now done.

1942-44 were lost to the second world war.

I had to give myself a couple of breaks between the posts.
 
stemline said:
All my stuff is VERY Hawthorn focused, so probably not much value to the collective here. But I guess some of it might plug a gap or two so I'll pop some stuff in this thread if it looks like it might relate to other projects mentioned here.

Starting with Hawthorn Subs
Click to expand...
Hi stemline - this stuff is amazing. Would you be okay with me incorporating this into my site? Things like player positions, captains, vice-captains, and so much more I'm very interested in. I'm just on my phone but I recall seeing you post a complete player position spreadsheet too and I'm happy to incorporate that. If you have VFA years stuff I can also incorporate that. Feel free to shoot me a message if you want discuss further. I've already been bringing in the works of Bassoon and Brad_5668.
 
McPang said:
Hi stemline - this stuff is amazing. Would you be okay with me incorporating this into my site? Things like player positions, captains, vice-captains, and so much more I'm very interested in. I'm just on my phone but I recall seeing you post a complete player position spreadsheet too and I'm happy to incorporate that. If you have VFA years stuff I can also incorporate that. Feel free to shoot me a message if you want discuss further. I've already been bringing in the works of Bassoon and Brad_5668.
Click to expand...


What website is this?
 

Log in to remove this ad.

Hi RedmanWasHere - thanks for the interest. To keep it brief (though I am terrible at doing so) - it's a recent project of mine I started for kicks that grew faster than I thought I would. I released a v1 in 2022 that was only VFL/AFL when I got back into footy and then created a v2 late last year when I decided it wasn't enough, expanded to state leagues, and had to re-design the database model and structural design of the site. It's nothing fancy; low-fidelity, low cost, and it would've even been free to host had I not added some additional features for security. Its main purpose is try to unify and consolidate as much information across the leagues as possible.

The current scope (having to draw the line somewhere - for now) includes, but is not limited to:
  • Competitions - not likely to expand in the near future while I try to complete this list:
    • VFL/AFL (1897-present)
    • VFA/VFL (1897-present, Div 1 only)
    • SANFL (1897-present inc. war merger years)
    • WAFL (1898-present)
    • TFL/TSL (1908-present)
    • Carnivals (1908-present, Section A or Section A/B only)
    • VFL/AFL Reserves (1919-1999 + present if it comes back)
    • VFA/VFL Reserves (1928-2017)
    • SANFL Reserves (1919-present)
    • WAFL Reserves (1925-present)
    • TFL/TSL Reserves (1928-present)
  • Match-level information:
    • Players (inc. whatever tallies e.g. total goals, behinds, disposals etc.)
    • Match-day coaches (inc. caretaker/non-caretaker)
    • Match-day captains (inc. acting/non-acting)
    • Match-day vice-captains (inc. acting/non-acting)
    • Player positions
    • Quarterly scores
    • Scoring timelines (i.e. score-by-score sequence e.g. player A scored a goal, then player B scored a behind etc.)
    • Venue
    • Overturned results via protest/ruling
    • Player substitutions
    • Umpires (field, goal, boundary)
    • Emergency umpire + replacements/substitutions
    • Brownlow votes
    • AFLCA Champion votes
    • Norm Smith votes
    • AFLCA Gary Ayres votes
  • Teams:
    • Home grounds each season
    • Leaders (captains, vice-captains, deputy vice-captains, leadership group) each season inc. noting changes mid-season
  • Drafts:
    • All drafts 1981-onwards excluding pre-draft/post-draft (i.e. a pick number must exist against the draftee)
  • Awards:
    • All-Australian
    • VFL/AFL teams of the year
    • VFA/VFL teams of the year
    • SANFL teams of the year
    • WAFL teams of the year
    • TSL teams of the year
  • Person-level details:
    • Names and aliases (not to be confused with nicknames)
    • Date of birth, date of death, place of birth, place of death
    • All links between player/coach/draftee/umpire i.e. (player-coach, player-draftee, player-umpire, coach-draftee, coach-umpire, draftee-umpire)
  • Other things, can't think of them at the moment
It's rare to come across the site because I haven't published the page publicly, nor set it to be crawled by Google. The main reason is because it's still somewhat a work-in-progress. It has its flaws and has its fair share of gaps that in some sense technically make it inaccurate in cases. The reason is because the database model design follows third normal form (3NF) (to a high degree) - this is to avoid conflicts. I would go into the details of the major issues this causes from a site perspective, but I've already made ths post too long, but an example is that I would have the incorrect number of games a Glenelg player has played for Glenelg, in some cases where records are incomplete. The short version is that it's a self-consistent site, but should never be linked to via URL, with information that can be misunderstood by generative AI models trying to learn from the internet (e.g. ChatGPT). The site was at first just meant to be a web-interface for me to analyse/interact with the data I had - 95% spaghetti, 5% care, 100% self-built end-to-end. Feel free to message me to discuss further. TLDR: Happy to share the site details via DM, just don't want misinformation from the site spreading and accidentally self-reinforcing in a loop.

But if you're curious about things the site includes, and the degree of scope, here are some examples:

1. How the teams lined up for the 2023 TSL grand final between Kingborough and North Launceston (which is why I'd love to incorporate the Hawthorn positional data)

1726326795368.png

2. The umpires for that same game

1726326917055.png

3. Algorithmically-determined AFLCA votes for a random AFL game

1726327263644.png
 
Last edited:
McPang said:
Hi stemline - this stuff is amazing. Would you be okay with me incorporating this into my site? Things like player positions, captains, vice-captains, and so much more I'm very interested in. I'm just on my phone but I recall seeing you post a complete player position spreadsheet too and I'm happy to incorporate that. If you have VFA years stuff I can also incorporate that. Feel free to shoot me a message if you want discuss further. I've already been bringing in the works of Bassoon and Brad_5668.
Click to expand...
You're free to use anything I've posted here, sure.
 
Thanks mate - I might reach out to you if I find anything that might need clarification (e.g. my data is out of date), but I'll try to incorporate it ASAP and will share the results.
 
stemline said:
You're free to use anything I've posted here, sure.
Click to expand...
Hawthorn FC positional data and substitution details now loaded for 1925-2023. Massive thanks for letting me use it. I'll be prioritising filling in the gaps I've got in TFL history now so I can then get through the player/captain/coach history e.g. Albert Collier coaching Cananore.

EDIT: TFL now complete for 1908-1915, 1919-1941, 1945-2000, 2009-2023. Coaching history (match-day) for New Town / Glenorchy inc. caretakers complete.
 
Last edited:

Remove this Banner Ad

Research Data gaps + big projects

Remove this Banner Ad

Back
Top