I would love to develop a more automated process when i write the posts for the for Small Vod Thread, and the ability to locate the frames with the image of the BW lobby in a long youtube video would be an interesting time saver...
The Big Programming Thread - Page 1015
Forum Index > General Forum |
Thread Rules 1. This is not a "do my homework for me" thread. If you have specific questions, ask, but don't post an assignment or homework problem and expect an exact solution. 2. No recruiting for your cockamamie projects (you won't replace facebook with 3 dudes you found on the internet and $20) 3. If you can't articulate why a language is bad, don't start slinging shit about it. Just remember that nothing is worse than making CSS IE6 compatible. 4. Use [code] tags to format code blocks. | ||
Burned Toast
Canada2040 Posts
I would love to develop a more automated process when i write the posts for the for Small Vod Thread, and the ability to locate the frames with the image of the BW lobby in a long youtube video would be an interesting time saver... | ||
WarSame
Canada1950 Posts
| ||
Silvanel
Poland4601 Posts
| ||
Nesserev
Belgium2760 Posts
| ||
Acrofales
Spain17186 Posts
| ||
emperorchampion
Canada9495 Posts
On January 07 2020 22:56 Nesserev wrote: Doing everything with selenium sounds like a massive pain and limiting your options; just use Google's Youtube Data API to check for new videos, download the videos, and then process the videos locally.
Btw, wouldn't it be better to search for the countdown signal? It's very distinct, and should be easy to recognize. Nice idea! In either the audio or visual route you can use ffmpeg (www.ffmpeg.org). The doot doot doot swwsssh audio is probably faster rather than extracting all the frames then searching through one-by-one. | ||
Djzapz
Canada10681 Posts
I just started dabbling in Python 3 days ago. It's my first experience "coding" it and I've been struggling so hard. I wrote a game where you start with $100 and then you gamble with it until you're at $0. I also made a thing that calculates factorials and displays n numbers of the fibonacci sequence. My code is absolute patchwork though. I'd like to be able to write scripts to parse through websites someday but damn, this is hard... | ||
tofucake
Hyrule18773 Posts
Parsing most sites isn't even usually worth it, as there are typically APIs to get the raw and specific data you want from the source, and if not, someone else has already written a scraper and made an API that'll give you that anyway. Unless you want to get into something like adblock tech or the like, scraping is a path to the dark side of HTML. | ||
Djzapz
Canada10681 Posts
On January 08 2020 13:49 tofucake wrote: Parsing websites is fucking awful. Ignoring the part where bad devs make everything 43 nested tables and good devs make things properly but every time a site is deployed every id and class name changes, it's fucking hard. Parsing most sites isn't even usually worth it, as there are typically APIs to get the raw and specific data you want from the source, and if not, someone else has already written a scraper and made an API that'll give you that anyway. Unless you want to get into something like adblock tech or the like, scraping is a path to the dark side of HTML. Damn . Well it's fun anyway! | ||
Deleted User 3420
24492 Posts
Djzapz if you need any python help I'd love to help you with questions. You can pm me here or some quicker alternative too if you need. | ||
tofucake
Hyrule18773 Posts
| ||
Djzapz
Canada10681 Posts
| ||
WarSame
Canada1950 Posts
Say their event is from 2019-05-03 to 2019-05-08 they would have 1 entry for each of those dates. I am trying to get the Id, and EventDate with a query without tons of subqueries/joins. Is there some efficient way of doing this that I'm missing? | ||
tofucake
Hyrule18773 Posts
| ||
Acrofales
Spain17186 Posts
There, that should help you in your way. Word of caution, that table sounds rather weird... this won't give you the last time an event was entered by a single person, it'll give you the last time any event was entered for a given EventDate by anybody. You won't be able to find different events on a single day, or anything about number of participants with the table as you specified it. | ||
Manit0u
Poland17046 Posts
On January 10 2020 07:34 WarSame wrote: Say you have a table with 3 important columns for some event, Id, CreationDate, and EventDate. Let's say that there are multiple entries for one person at one event, and a new entry gets created every time their information changes. How can you use SQL to grab the most recent Id and EventDate from them for each EventDate? Say their event is from 2019-05-03 to 2019-05-08 they would have 1 entry for each of those dates. I am trying to get the Id, and EventDate with a query without tons of subqueries/joins. Is there some efficient way of doing this that I'm missing? Not entirely sure what do you want to do. What's creation date? Is Id an uuid or a bigint? Is it auto-increment PK or FK? If it's what I assume it is you'll have some kind of user/customer table and event table that belongs to the user/customer. In that case it's really easy.
This gets you the most recent entry for specific customer. What you described seems rather odd and super inefficient though. Why not just store event dates as daterange? Why create new entries upon updates if you can just update the old ones? Edit: Let me rephrase it a bit... There must be some way to tell which event dates belong to which group (especially that it seems you can have multiple events for the same date). Also, with the idea of creating new entries when something is changed there must be a way to tell which dates have been invalidated, what is it? All this info can help with getting what you want from the table. | ||
Acrofales
Spain17186 Posts
I suggest either redesigning your DB a bit, or giving us more information, because as is, it doesn't make sense. | ||
WarSame
Canada1950 Posts
I can tie the Event Snapshot(1 day of an event) to a customer, so that part I'm not worried about. What I'm trying to do is find the most recent updates to the event before a certain date cutoff. Say I'm looking before the end of March, I want to find the last Event Snapshot at the end of March. So I need to pull out the most recent Event Snapshot for that Event, but I need to do so for each date of the Event(so I can sum up revenues of the event, for example). @Acrofales, thanks! Our DB on the backend is SQL Server but I'm trying to do this in SalesForce Developer Console(bleh) so it doesn't translate directly. I asked about SQL because I figured it was the most familiar to everyone but now I see the flaw in my plan. Do you know of a way without using any tailored SQL solution? i.e. system agnostic, using just joins and the like. | ||
Manit0u
Poland17046 Posts
On January 11 2020 01:23 WarSame wrote: It's not my DB and I can't redesign it but I have the same questions as y'all :D I can tie the Event Snapshot(1 day of an event) to a customer, so that part I'm not worried about. What I'm trying to do is find the most recent updates to the event before a certain date cutoff. Say I'm looking before the end of March, I want to find the last Event Snapshot at the end of March. So I need to pull out the most recent Event Snapshot for that Event, but I need to do so for each date of the Event(so I can sum up revenues of the event, for example). @Acrofales, thanks! Our DB on the backend is SQL Server but I'm trying to do this in SalesForce Developer Console(bleh) so it doesn't translate directly. I asked about SQL because I figured it was the most familiar to everyone but now I see the flaw in my plan. Do you know of a way without using any tailored SQL solution? i.e. system agnostic, using just joins and the like. Ok, so there's a way to group events by customer. Is there a way to also group them by event (multiple snapshots being a group for continuous multi-day event)? And what about invalidated snapshots like I mentioned? That's all you need really to make it work. | ||
Acrofales
Spain17186 Posts
On January 10 2020 07:34 WarSame wrote: Say you have a table with 3 important columns for some event, Id, CreationDate, and EventDate. Let's say that there are multiple entries for one person at one event, and a new entry gets created every time their information changes. How can you use SQL to grab the most recent Id and EventDate from them for each EventDate? Say their event is from 2019-05-03 to 2019-05-08 they would have 1 entry for each of those dates. I am trying to get the Id, and EventDate with a query without tons of subqueries/joins. Is there some efficient way of doing this that I'm missing? Okay, without using SQL: well, you can easily find the max(creationDate) in a loop over all the entries. Something like (in python, but converting to whatever language you like is easy):
This way, latest_ids has all the ids of the last creations for each event date. It's not too inefficient, but if your DB has a LOT of entries, then you do need to move this logic to where the data is being retrieved (the query). If you don't have access then I don't really know what you can do to speed it up. I still don't really understand the point of this: if two events coincide in their event_date you're still screwed. But I guess that is simply impossible in whatever use case this structure is being used for. | ||
| ||