• Log InLog In
  • Register
Liquid`
Team Liquid Liquipedia
EST 11:43
CET 17:43
KST 01:43
  • Home
  • Forum
  • Calendar
  • Streams
  • Liquipedia
  • Features
  • Store
  • EPT
  • TL+
  • StarCraft 2
  • Brood War
  • Smash
  • Heroes
  • Counter-Strike
  • Overwatch
  • Liquibet
  • Fantasy StarCraft
  • TLPD
  • StarCraft 2
  • Brood War
  • Blogs
Forum Sidebar
Events/Features
News
Featured News
[ASL22] Finals Preview: Sweet Victory!12[ASL22] Ro4 Preview: Mirror Mirror6[ASL22] Ro8 Preview: Within Reach5[ASL22] Ro8 Preview: In A Tizzy11[ASL22] Ro16 Preview: Holy Diver5
Community News
Weekly Cups (Nov 9-15): Clem and herO take over0Dark completes military service, returns to SC212[BSL23] RO16 Group Stage20Weekly Cups (Nov 2-8): herO back on top?2Weekly Cups (Oct 26 - Nov 1): MaxPax, herO, Solar win0
StarCraft 2
General
No more SC2 streamers on team liquid? Weekly Cups (Nov 9-15): Clem and herO take over Dark completes military service, returns to SC2 Serral's getting trashed by New Catsle's Mind Weekly Cups (Nov 2-8): herO back on top?
Tourneys
RSL goes to London! 2026 Offline Finals Nov 21-22 Sparkling Tuna Cup - Weekly Open Tournament Sea Duckling Open (Global, Bronze-Diamond) Dark returns! GSL CK #7 4v4 Team Battle (Nov 19) soO & FanTaSy return! SC2 Kevin Scrim 4v4 (Nov 16)
Strategy
[H] ZvP Mid-Late Game: Stalkers Collossi HT
Custom Maps
[G] How to playtest arcade maps Starcraft 2 Advance [D] the control group Simple Questions/Answers
External Content
Mutation # 552 Secret Storm The PondCast: SC2 News & Results Mutation # 551 Robotic Revival Mutation # 550 Graveyard Shift
Brood War
General
List of top players active on ShieldBattery BW General Discussion NaDa's Body Shield Battery Server New Patch BGH Auto Balance -> http://bghmmr.eu/
Tourneys
SC1 Kevin Scrim hosted by SOOP (Nov 18-19) [IPSL] Winter 26-27 Round of 16! [Megathread] Daily Proleagues 2v2v2v2 Tournament Stage 2: Players
Strategy
Pluto's TvP 5Marine 2Medic Push Simple Questions, Simple Answers Infested Terran strategies in 2026 Nuke strategy worked in K-League 2026-11-09
Other Games
General Games
Nintendo Switch Thread Stormgate/Frost Giant Megathread Free Space RTS Game, optimised f. Phone/Tablet etc List of Game Threads The PlayStation 5
Dota 2
League of Legends
[TL LoL EUW IHs] Teemo shall perish
Heroes of the Storm
Heroes of the Storm 2.0
Hearthstone
Deck construction bug
TL Mafia
TL Mafia Community Thread
Community
General
US Politics Mega-thread Things Aren’t Peaceful in Palestine Russo-Ukrainian War Thread Artificial Intelligence Thread Trading/Investing Thread
Fan Clubs
Fan Club Resource Thread Reynor Fan Club The Flash Fan Club
Media & Entertainment
Movie Discussion! Anime Discussion Thread [Manga] One Piece
Sports
TeamLiquid Health and Fitness Initiative For 2023 Football (Soccer) Thread
World Cup 2022
Tech Support
Computer Build, Upgrade & Buying Resource Thread
TL Community
The Automated Ban List Recent Gifted Posts
Blogs
The time I organised my…
AssyrianKing
Why I built Repalyzer (1.1.…
BlueStar
Innovate or Game Over?
TrAiDoS
ShieldBattery on Mac
Kraekkling
Can Bots Beat Pros?? Starcr…
namkraft
StarCraft improvement
iopq
Socialism Anyone?
GreenHorizons
Customize Sidebar...

Website Feedback

Closed Threads



Active: 11560 users

The Big Programming Thread - Page 1015

Forum Index > General Forum
Post a Reply
Prev 1 1013 1014 1015 1016 1017 1032 Next
Thread Rules
1. This is not a "do my homework for me" thread. If you have specific questions, ask, but don't post an assignment or homework problem and expect an exact solution.
2. No recruiting for your cockamamie projects (you won't replace facebook with 3 dudes you found on the internet and $20)
3. If you can't articulate why a language is bad, don't start slinging shit about it. Just remember that nothing is worse than making CSS IE6 compatible.
4. Use [code] tags to format code blocks.
Burned Toast *
Profile Blog Joined May 2007
Canada2040 Posts
January 07 2020 01:56 GMT
#20281
Is there an API or something that can locate a specific frame in a Youtube video by image recognition ?

I would love to develop a more automated process when i write the posts for the for Small Vod Thread, and the ability to locate the frames with the image of the BW lobby in a long youtube video would be an interesting time saver...
TvT matchup is sometimes worse than jailtime
WarSame
Profile Blog Joined February 2010
Canada1950 Posts
January 07 2020 02:48 GMT
#20282
Could you not download the video then run some sort of recognition software on that?
Can it be I stayed away too long? Did you miss these rhymes while I was gone?
Silvanel
Profile Blog Joined March 2003
Poland4770 Posts
January 07 2020 08:13 GMT
#20283
I think its more a problem with recognition itself? If not You can always automate the browser with something like selenium and simply run recognition on what is displayed to the user. Unless of course You want to run it headless and/on multiple videos at the same time. Then its becomes a little more complicated but still doable with something like selenium grid.
Pathetic Greta hater.
Nesserev
Profile Blog Joined January 2011
Belgium2760 Posts
Last Edited: 2020-01-07 13:56:33
January 07 2020 13:56 GMT
#20284
--- Nuked ---
Acrofales
Profile Joined August 2010
Spain18501 Posts
Last Edited: 2020-01-07 18:42:29
January 07 2020 18:41 GMT
#20285
Agree with nesserev. Doing any kind of serious processing through selenium sounds like a huge pain in the ass. However awesome selenium is, it isn't really designed for this type of task. I also don't think the youtube data api allows you to download a video, but there are tons of ways of doing that (and automating this process if you have a URL, which you *can* search for with the data API).
emperorchampion
Profile Blog Joined December 2008
Canada9496 Posts
January 07 2020 22:15 GMT
#20286
On January 07 2020 22:56 Nesserev wrote:
Show nested quote +
On January 07 2020 17:13 Silvanel wrote:
I think its more a problem with recognition itself? If not You can always automate the browser with something like selenium and simply run recognition on what is displayed to the user. Unless of course You want to run it headless and/on multiple videos at the same time. Then its becomes a little more complicated but still doable with something like selenium grid.

Doing everything with selenium sounds like a massive pain and limiting your options; just use Google's Youtube Data API to check for new videos, download the videos, and then process the videos locally.

  1. uses database to keep track of relevant channels and their videos
  2. periodically scan channel for new videos
  3. download videos
  4. use visual/audio recognition library to determine timestamps


Btw, wouldn't it be better to search for the countdown signal? It's very distinct, and should be easy to recognize.


Nice idea!

In either the audio or visual route you can use ffmpeg (www.ffmpeg.org). The doot doot doot swwsssh audio is probably faster rather than extracting all the frames then searching through one-by-one.
TRUEESPORTS || your days as a respected member of team liquid are over
Djzapz
Profile Blog Joined August 2009
Canada10681 Posts
Last Edited: 2020-01-08 01:32:43
January 08 2020 01:29 GMT
#20287
Probably not the point of this thread but I'll just write what I've been up to.

I just started dabbling in Python 3 days ago. It's my first experience "coding" it and I've been struggling so hard. I wrote a game where you start with $100 and then you gamble with it until you're at $0. I also made a thing that calculates factorials and displays n numbers of the fibonacci sequence. My code is absolute patchwork though.

I'd like to be able to write scripts to parse through websites someday but damn, this is hard...
"My incompetence with power tools had been increasing exponentially over the course of 20 years spent inhaling experimental oven cleaners"
tofucake
Profile Blog Joined October 2009
Hyrule19272 Posts
January 08 2020 04:49 GMT
#20288
Parsing websites is fucking awful. Ignoring the part where bad devs make everything 43 nested tables and good devs make things properly but every time a site is deployed every id and class name changes, it's fucking hard.

Parsing most sites isn't even usually worth it, as there are typically APIs to get the raw and specific data you want from the source, and if not, someone else has already written a scraper and made an API that'll give you that anyway.

Unless you want to get into something like adblock tech or the like, scraping is a path to the dark side of HTML.
Liquipediaasante sana squash banana
Djzapz
Profile Blog Joined August 2009
Canada10681 Posts
January 08 2020 05:01 GMT
#20289
On January 08 2020 13:49 tofucake wrote:
Parsing websites is fucking awful. Ignoring the part where bad devs make everything 43 nested tables and good devs make things properly but every time a site is deployed every id and class name changes, it's fucking hard.

Parsing most sites isn't even usually worth it, as there are typically APIs to get the raw and specific data you want from the source, and if not, someone else has already written a scraper and made an API that'll give you that anyway.

Unless you want to get into something like adblock tech or the like, scraping is a path to the dark side of HTML.

Damn . Well it's fun anyway!
"My incompetence with power tools had been increasing exponentially over the course of 20 years spent inhaling experimental oven cleaners"
Deleted User 3420
Profile Blog Joined May 2003
24492 Posts
Last Edited: 2020-01-08 19:51:22
January 08 2020 17:33 GMT
#20290
Some websites can be pretty atrocious, but in general my experience with scraping websites is that it isn't *too* awful. A bit menial but how horrible it is will generally correspond mostly to how varied the information you are trying to get is.

Djzapz if you need any python help I'd love to help you with questions. You can pm me here or some quicker alternative too if you need.
tofucake
Profile Blog Joined October 2009
Hyrule19272 Posts
January 08 2020 19:11 GMT
#20291
Yeah python is the easy part, I can help with that too.
Liquipediaasante sana squash banana
Djzapz
Profile Blog Joined August 2009
Canada10681 Posts
January 08 2020 21:09 GMT
#20292
Thanks guys, I appreciate the offer! I'm good for now because my wife is an IT project manager and while she hasn't worked with Python very much, she knows enough that she can answer any questions I have for now. Cheers
"My incompetence with power tools had been increasing exponentially over the course of 20 years spent inhaling experimental oven cleaners"
WarSame
Profile Blog Joined February 2010
Canada1950 Posts
January 09 2020 22:34 GMT
#20293
Say you have a table with 3 important columns for some event, Id, CreationDate, and EventDate. Let's say that there are multiple entries for one person at one event, and a new entry gets created every time their information changes. How can you use SQL to grab the most recent Id and EventDate from them for each EventDate?

Say their event is from 2019-05-03 to 2019-05-08 they would have 1 entry for each of those dates. I am trying to get the Id, and EventDate with a query without tons of subqueries/joins. Is there some efficient way of doing this that I'm missing?
Can it be I stayed away too long? Did you miss these rhymes while I was gone?
tofucake
Profile Blog Joined October 2009
Hyrule19272 Posts
Last Edited: 2020-01-10 02:27:08
January 10 2020 02:26 GMT
#20294
oops saw the example
Liquipediaasante sana squash banana
Acrofales
Profile Joined August 2010
Spain18501 Posts
January 10 2020 06:31 GMT
#20295
https://stackoverflow.com/questions/7118170/sql-server-select-only-the-rows-with-maxdate

There, that should help you in your way.

Word of caution, that table sounds rather weird... this won't give you the last time an event was entered by a single person, it'll give you the last time any event was entered for a given EventDate by anybody. You won't be able to find different events on a single day, or anything about number of participants with the table as you specified it.
Manit0u
Profile Blog Joined August 2004
Poland17875 Posts
Last Edited: 2020-01-10 12:27:11
January 10 2020 12:08 GMT
#20296
On January 10 2020 07:34 WarSame wrote:
Say you have a table with 3 important columns for some event, Id, CreationDate, and EventDate. Let's say that there are multiple entries for one person at one event, and a new entry gets created every time their information changes. How can you use SQL to grab the most recent Id and EventDate from them for each EventDate?

Say their event is from 2019-05-03 to 2019-05-08 they would have 1 entry for each of those dates. I am trying to get the Id, and EventDate with a query without tons of subqueries/joins. Is there some efficient way of doing this that I'm missing?


Not entirely sure what do you want to do. What's creation date? Is Id an uuid or a bigint? Is it auto-increment PK or FK?

If it's what I assume it is you'll have some kind of user/customer table and event table that belongs to the user/customer. In that case it's really easy.


SELECT Id, CustomerId, EventDate FROM events
WHERE CustomerId = someId
ORDER BY EventDate DESC
LIMIT 1


This gets you the most recent entry for specific customer.

What you described seems rather odd and super inefficient though. Why not just store event dates as daterange? Why create new entries upon updates if you can just update the old ones?

Edit: Let me rephrase it a bit...

There must be some way to tell which event dates belong to which group (especially that it seems you can have multiple events for the same date). Also, with the idea of creating new entries when something is changed there must be a way to tell which dates have been invalidated, what is it? All this info can help with getting what you want from the table.
Time is precious. Waste it wisely.
Acrofales
Profile Joined August 2010
Spain18501 Posts
January 10 2020 13:57 GMT
#20297
Oh, I missed that an event is multiple days, and therefore has multiple entries in this table. In that case the situation is even more confusing.

I suggest either redesigning your DB a bit, or giving us more information, because as is, it doesn't make sense.
WarSame
Profile Blog Joined February 2010
Canada1950 Posts
January 10 2020 16:23 GMT
#20298
It's not my DB and I can't redesign it but I have the same questions as y'all :D

I can tie the Event Snapshot(1 day of an event) to a customer, so that part I'm not worried about. What I'm trying to do is find the most recent updates to the event before a certain date cutoff. Say I'm looking before the end of March, I want to find the last Event Snapshot at the end of March. So I need to pull out the most recent Event Snapshot for that Event, but I need to do so for each date of the Event(so I can sum up revenues of the event, for example).

@Acrofales, thanks! Our DB on the backend is SQL Server but I'm trying to do this in SalesForce Developer Console(bleh) so it doesn't translate directly. I asked about SQL because I figured it was the most familiar to everyone but now I see the flaw in my plan. Do you know of a way without using any tailored SQL solution? i.e. system agnostic, using just joins and the like.
Can it be I stayed away too long? Did you miss these rhymes while I was gone?
Manit0u
Profile Blog Joined August 2004
Poland17875 Posts
January 10 2020 17:30 GMT
#20299
On January 11 2020 01:23 WarSame wrote:
It's not my DB and I can't redesign it but I have the same questions as y'all :D

I can tie the Event Snapshot(1 day of an event) to a customer, so that part I'm not worried about. What I'm trying to do is find the most recent updates to the event before a certain date cutoff. Say I'm looking before the end of March, I want to find the last Event Snapshot at the end of March. So I need to pull out the most recent Event Snapshot for that Event, but I need to do so for each date of the Event(so I can sum up revenues of the event, for example).

@Acrofales, thanks! Our DB on the backend is SQL Server but I'm trying to do this in SalesForce Developer Console(bleh) so it doesn't translate directly. I asked about SQL because I figured it was the most familiar to everyone but now I see the flaw in my plan. Do you know of a way without using any tailored SQL solution? i.e. system agnostic, using just joins and the like.


Ok, so there's a way to group events by customer. Is there a way to also group them by event (multiple snapshots being a group for continuous multi-day event)? And what about invalidated snapshots like I mentioned? That's all you need really to make it work.
Time is precious. Waste it wisely.
Acrofales
Profile Joined August 2010
Spain18501 Posts
January 11 2020 12:49 GMT
#20300
On January 10 2020 07:34 WarSame wrote:
Say you have a table with 3 important columns for some event, Id, CreationDate, and EventDate. Let's say that there are multiple entries for one person at one event, and a new entry gets created every time their information changes. How can you use SQL to grab the most recent Id and EventDate from them for each EventDate?

Say their event is from 2019-05-03 to 2019-05-08 they would have 1 entry for each of those dates. I am trying to get the Id, and EventDate with a query without tons of subqueries/joins. Is there some efficient way of doing this that I'm missing?

Okay, without using SQL:

well, you can easily find the max(creationDate) in a loop over all the entries. Something like (in python, but converting to whatever language you like is easy):

latest_creations = defaultdict(int)
latest_ids = defaultdict(int)
for e_id, creation_date, event_date in entries:
latest = latest_creations[event_date]
entry_time = convert_to_timestamp(creation_date)
if latest < entry_time:
latest_creations[event_date] = entry_time
latest_ids[event_date] = e_id


This way, latest_ids has all the ids of the last creations for each event date. It's not too inefficient, but if your DB has a LOT of entries, then you do need to move this logic to where the data is being retrieved (the query). If you don't have access then I don't really know what you can do to speed it up.

I still don't really understand the point of this: if two events coincide in their event_date you're still screwed. But I guess that is simply impossible in whatever use case this structure is being used for.
Prev 1 1013 1014 1015 1016 1017 1032 Next
Please log in or register to reply.
Live Events Refresh
Next event in 17m
[ Submit Event ]
Live Streams
Refresh
StarCraft 2
Serral 1752
ProTech153
TKL 67
StarCraft: Brood War
Britney 32403
Calm 4054
Horang2 406
Soulkey 236
firebathero 233
Rush 180
ggaemo 137
Pusan 56
Movie 27
Trap 27
[ Show more ]
Terrorterran 25
Killer 20
Bishop 16
Rock 15
scan(afreeca) 13
sorry 12
Sacsri 10
yabsab 8
Dota 2
Gorgc2920
qojqva2405
monkeys_forever380
Counter-Strike
gobbtv163
edward52
Other Games
Liquid`RaSZi1300
B2W.Neo615
hiko597
Lowko334
crisheroes266
Liquid`VortiX166
ArmadaUGS141
Trikslyr52
QueenE33
KnowMe31
UpATreeSC30
markeloff27
[ Show 12 non-featured ]
StarCraft 2
• AfreecaTV YouTube
• intothetv
• Kozan
• IndyKCrew
• Migwel
StarCraft: Brood War
• mYiSmile119
• BSLYoutube
• STPLYoutube
• ZZZeroYoutube
Dota 2
• WagamamaTV535
League of Legends
• TFBlade1008
Other Games
• Shiphtur483
Upcoming Events
Monday Night Weeklies
17m
PiGosaur Cup
8h 17m
The PondCast
17h 17m
University League
17h 17m
LiuLi Cup
18h 17m
Rogue vs TBD
Maru vs TBD
OSC
23h 17m
Replay Cast
1d 6h
GSL
1d 18h
Maru vs Dark
Patches Events
1d 21h
OSC
1d 23h
[ Show More ]
Replay Cast
2 days
KCM Race Survival
2 days
Ultimate Battle
2 days
ZerO vs Bisu
RSL Revival
3 days
BSL
4 days
eOnzErG vs StRyKeR
Cross vs JDConan
RSL Revival
4 days
IPSL
5 days
Ret vs ggnore13
nOmaD vs Aether
BSL
5 days
Bonyth vs Doodle
Kyrie vs Jimin
Replay Cast
5 days
WardiTV Weekly
5 days
Kung Fu Cup
6 days
Solar vs Classic
ByuN vs SHIN
Liquipedia Results

Completed

Proleague 2026-11-16
Blizzard Classic Cup 2026
HCC Season 3

Ongoing

Acropolis #5
KCM Race Survival 2026 Season 4
International ShieldBattery League
BSL Season 23
IPSL Winter 2026-27
Calamity Cup Div. A Season 8
Underdog Cup 2026 Div 2-3
Stake Ranked Episode 6
BLAST Rivals Fall 2026
IEM Beijing 2026
PGL Masters Bucharest 2026
Stake Ranked Episode 5
1win Private Club #2
Thunderpick World Champ. '26
ESL Pro League Season 24
Stake Ranked Episode 4
1win Private Club #1
Logitech G Play Connect 2026
SL StarSeries Fall 2026
FISSURE Playground #3
BLAST Open Fall 2026

Upcoming

Sector 591
YSL S4
SC4ALL II: Brood War
NBSL 2026 Xiamen LAN
BSL 23: Non-Korean Championship
HSC XXX
Stellar Fest 2: Lunar Cup
StarCraft2 Weekly Open Tournament 2026 Grand Finals
SC4ALL II: StarCraft II
Kung Fu Cup 2026 Grand Finals
RSL Offline Finals
IEM Kraków 2027
PGL Bucharest 2027
BLAST Bounty Winter 2027
PGL Major Singapore 2026
TLPD

1. ByuN
2. TY
3. Dark
4. Solar
5. Stats
6. Nerchio
7. sOs
8. soO
9. INnoVation
10. Elazer
1. Rain
2. Flash
3. EffOrt
4. Last
5. Bisu
6. Soulkey
7. Mini
8. Sharp
Sidebar Settings...

Advertising | Privacy Policy | Terms Of Use | Contact Us

Original banner artwork: Jim Warren
The contents of this webpage are copyright © 2026 TLnet. All Rights Reserved.