| Forum Index > General Forum |
Thread Rules 1. This is not a "do my homework for me" thread. If you have specific questions, ask, but don't post an assignment or homework problem and expect an exact solution. 2. No recruiting for your cockamamie projects (you won't replace facebook with 3 dudes you found on the internet and $20) 3. If you can't articulate why a language is bad, don't start slinging shit about it. Just remember that nothing is worse than making CSS IE6 compatible. 4. Use [code] tags to format code blocks. | ||
zatic
Zurich15363 Posts
On May 27 2020 10:47 travis wrote: That's actually more complicated than I need. The game won't have any spatial aspects, so no movement. Just conversation, so just text output and maybe some input. Python is totally fine for that then. This sounds like a really simple flask or bottle app? | ||
|
Yurie
12058 Posts
And since I fully missed the web part of the requirement and don't want to look stupid. https://renpy.beuc.net/ | ||
|
Deleted User 3420
24492 Posts
 thanks everyone | ||
|
JimmyJRaynor
Canada17329 Posts
On May 07 2020 19:47 Manit0u wrote: On another note, I'll have some fun in one side project now. Will be working with C code that was last touched in 1998. how is it going so far? I love these types of situations in the software development industry... that is, going through ancient code. I've been doing the hermit-coder//"Bob Fitch" thing for several weeks now. A nursing org is replacing/overhauling a Contact Tracing Application made with CA-Clipper in the 1990s. The system's last big update occurred in 2005. It had major upgrades from 1995 to 2005. There is zero documentation. The last programmer who worked on the system is in his 60s. He experienced a major health catastrophe six months ago. I get what info I can from him. He charges the nursing org $200/hour so my time with him is very limited. There is one business analyst still working at the nursing organization that used this application. She has a vague recollection of how the system worked. My best source of info : Turn on the Tracer and step through the code one line at a time. Hermit Coder War Stories are the best... Bob Fitch rescued the Starcraft 1 project by locking himself in a room for several weeks/months and making the SC1 engine. I've been doing a similar thing for the last two months. I've been tasked with jamming together a project in a very unreasonably short timeline. I've done the hermit coder thing several times in my career. It has built me some major street cred with project managers and CIOs. Some people can't handle it for whatever reason. I'm an only child and grew up only with my mom. As a child and teenager I had the whole house to myself a lot. So doing the hermit coder thing is right in my comfort zone. I hear people whine, bitch, moan and complain about "crunch time". Personally, I love it. I'm having a blast. On May 07 2020 03:27 Manit0u wrote: Unfortunately this is not something that's easily achievable yourself. Only after spending years doing similar projects over and over, maintaining them, revisiting them and changing them do you really begin to easily recognize all those common patterns and such. Sure, there are books on enterprise patterns etc. but you can memorize them and still not see the patterns in the actual code (and there are other patterns than just coding ones). Another thing with experience and doing the same stuff over and over is that you can see the pitfalls in advance. You know that you need to do extra bit of coding to prevent code locks, running out of memory, problems with garbage collection in certain situations and you know that before the first line of code is written which saves you a lot of time down the line (when you have to fix code that's been in production for a while and only then the problems start to show up). i find this an interesting theme brought up in Aldous Huxley's "Brave New World". One can memorize and one can even parrot something and still never actually understand. “The Nile is the longest river in Africa and the second in length of all the rivers of the globe. Although falling short of the length of the Mississippi-Missouri, the Nile is at the head of all rivers as regards the length of its basin, which extends through 35 degrees of latitude …” At breakfast the next morning, “Tommy,” some one says, “do you know which is the longest river in Africa?” A shaking of the head. “But don’t you remember something that begins: The Nile is the …” “The – Nile – is – the – longest – river – in – Africa – and – the – second -in – length – of – all – the – rivers – of – the – globe …” The words come rushing out. “Although – falling – short – of …” “Well now, which is the longest river in Africa?” The eyes are blank. “I don’t know.” “But the Nile, Tommy.” “The – Nile – is – the – longest – river – in – Africa – and – second …” “Then which river is the longest, Tommy?” Tommy burst into tears. “I don’t know,” he howls.) later in the same chapter... That howl, the Director made it plain, discouraged the earliest investigators. The experiments were abandoned. No further attempt was made to teach children the length of the Nile in their sleep. Quite rightly. You can’t learn a science unless you know what it’s all about. https://opentextbc.ca/englishliterature/chapter/brave-new-world-chapter-two/ | ||
|
Manit0u
Poland17692 Posts
On May 31 2020 21:58 JimmyJRaynor wrote: how is it going so far? I love these types of situations in the software development industry... that is, going through ancient code. It's going slow. I'm preoccupied with some other stuff so that's mostly a side-project for me. In any case, I've created new structure for the directory with subdirectories and such (previously it was just a mess with everything being in a single directory) and started cleaning it up a bit by removing unnecessary stuff that clogs it up (big chunks of the code were poached from somewhere else and half of it is unused). Once that's done I'll start the real refactoring. | ||
|
Chocolate
United States2350 Posts
On May 31 2020 21:58 JimmyJRaynor wrote: how is it going so far? I love these types of situations in the software development industry... that is, going through ancient code. I've been doing the hermit-coder//"Bob Fitch" thing for several weeks now. A nursing org is replacing/overhauling a Contact Tracing Application made with CA-Clipper in the 1990s. The system's last big update occurred in 2005. It had major upgrades from 1995 to 2005. There is zero documentation. The last programmer who worked on the system is in his 60s. He experienced a major health catastrophe six months ago. I get what info I can from him. He charges the nursing org $200/hour so my time with him is very limited. There is one business analyst still working at the nursing organization that used this application. She has a vague recollection of how the system worked. My best source of info : Turn on the Tracer and step through the code one line at a time. Hermit Coder War Stories are the best... Bob Fitch rescued the Starcraft 1 project by locking himself in a room for several weeks/months and making the SC1 engine. I've been doing a similar thing for the last two months. I've been tasked with jamming together a project in a very unreasonably short timeline. I've done the hermit coder thing several times in my career. It has built me some major street cred with project managers and CIOs. Some people can't handle it for whatever reason. I'm an only child and grew up only with my mom. As a child and teenager I had the whole house to myself a lot. So doing the hermit coder thing is right in my comfort zone. I hear people whine, bitch, moan and complain about "crunch time". Personally, I love it. I'm having a blast. i find this an interesting theme brought up in Aldous Huxley's "Brave New World". One can memorize and one can even parrot something and still never actually understand. later in the same chapter... https://opentextbc.ca/englishliterature/chapter/brave-new-world-chapter-two/ Great post. Crunch time is my favorite time too. I actually love hitting those weeklong streaks of huge productivity (and usually knowledge/skill) gains and making heads turn. However in a team setting it can work against you if peers get jealous/angry at you for setting higher standards which are admittedly very unsustainable or if managers decide they can squeeze more work out of the team by creating false urgency. The biggest issue with patterns is as you said, thinking that more applied patterns to code = better code because patterns are in books about writing software. Patterns when misapplied can also make things needlessly brittle or verbose and hard to understand. Even inheritance can be an antipattern. Lately I've been working a lot with an old-ish piece of cloud infrastructure written in C++ which over time morphed into an uber-dependency ball of mud surrounding a white-hot request path. It has a pattern of applying behavior to the inbound request and outbound response *everywhere*. If you want to make a change, you better get ready to read 100k lines of code, because you'll have no way of understanding how the change will actually impact a request. Sure, there are tests, but only the Old Ones understood the design well enough to test changes coherently (and even so most of the tests are just trivially verifying a boring implementation detail or extraneous behavior - many of which never see the light of day in real use cases, but which are kept around Just In Case), and there's no way to spin up anything remotely resembling an actual prod environment, because it has so many other infrastructural dependencies. I've learned so much of what not to do, or how something which sounds like a good design can degenerate over a decade/dozens of engineers, that it's been a great learning experience. Biggest takeaway: if you're doing something impressive, it doesn't matter how shitty or unmaintainable your code is as long as it gets you promoted into something that doesn't require you to maintain it. My VP is one of the original authors  | ||
|
WarSame
Canada1950 Posts
Does anyone know of a way to monitor the incoming/outgoing HTTP requests/responses? Currently I'm trying VPC Traffic Mirroring but it seems relatively involved. Is there a better/easier option? | ||
|
Acrofales
Spain18233 Posts
On June 10 2020 10:49 WarSame wrote: I have a DB instance on AWS RDS which is giving a crazy amount of data in replies. I want to see what the requests and replies are to see why the amount of data is so high. I've found this pretty difficult to do. Does anyone know of a way to monitor the incoming/outgoing HTTP requests/responses? Currently I'm trying VPC Traffic Mirroring but it seems relatively involved. Is there a better/easier option? I'm not all that familiar with all the AWS services, but there's cloudwatch that you can use to monitor most of them. There's also the log files themselves. And imho your best option is to find one of the clients that is causing the data, and look at what they're doing. Imho looking at what the server is doing or what the client is doing is almost always easier than trying to figure it out from packages on the network. Unless it's actually a network issue. | ||
|
Silvanel
Poland4742 Posts
| ||
|
JimmyJRaynor
Canada17329 Posts
On June 10 2020 10:49 WarSame wrote: I have a DB instance on AWS RDS which is giving a crazy amount of data in replies. I want to see what the requests and replies are to see why the amount of data is so high. I've found this pretty difficult to do. Does anyone know of a way to monitor the incoming/outgoing HTTP requests/responses? Currently I'm trying VPC Traffic Mirroring but it seems relatively involved. Is there a better/easier option? i use West Wind Web Surge. https://websurge.west-wind.com/ On June 06 2020 12:07 Chocolate wrote: The biggest issue with patterns is as you said, thinking that more applied patterns to code = better code because patterns are in books about writing software. Patterns when misapplied can also make things needlessly brittle or verbose and hard to understand. Even inheritance can be an antipattern. lol, yep. when i first wrapped my head around how inheritance worked i created an inheritance anti-pattern. i was just trying to fully understand it so i shoe-horned it in a place it did not belong. For me , it was a solution.. looking for a problem. regarding anti-patterns ... this book is old, however, imo, it is a timeless classic. you can probably find it for free someplace due to its advanced age. it is fucking hilarious. https://www.amazon.ca/AntiPatterns-Refactoring-Software-Architectures-Projects/dp/0471197130 | ||
|
WarSame
Canada1950 Posts
On June 10 2020 15:26 Acrofales wrote: I'm not all that familiar with all the AWS services, but there's cloudwatch that you can use to monitor most of them. There's also the log files themselves. And imho your best option is to find one of the clients that is causing the data, and look at what they're doing. Imho looking at what the server is doing or what the client is doing is almost always easier than trying to figure it out from packages on the network. Unless it's actually a network issue. Cloudwatch I believe is only used for metrics. It's what I used to tell me why we might be getting some latency for DB access. It showed me we have 2 GB/5 minutes with only a few users online. I was thinking that too but haven't been able to get DB access as easily as I hoped. I was reading up on it and most replies said that DBs generally have most of their queries saved, which would be helpful, but wouldn't be proof-positive that they are why the replies are so large. I don't know their DB and don't have great connectivity to their team so I was hoping to black box the DB, but I've requested access now. I tried VPC mirroring but after spending ~2 hours reading up on it and setting it up I was just not getting any traffic to my monitoring EC2 instance, so that was frustrating. | ||
tofucake
Hyrule19194 Posts
Try reading this for whatever DB they have running: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.html it might help... Otherwise, you'll probably need to do in-DB checks. Postgres has stl_* tables, Maria/MySQL have similar tables, and MSSQL has the same sort of thing but I forget what it's called. If you have AWS console access to the instance there's a tab for query logs in the cluster panel somewhere. | ||
|
WarSame
Canada1950 Posts
1. Tons of queries per conversation - ~75 queries to display one page to user. This is not a SPA and is stateless so the whole state must be reloaded every time 2. Queries with extra fields - a number of queries are using select * for tables with ~90 fields. Each field is ~100 bytes, leading to 10 KB replies. When you have 75 replies that's quite a lot. The solutions I've proposed: 1. reduce the number of queries used, use a SPA or something else to reduce it. Long term strategy. 2. don't select *, select the fields you need This should help reduce the traffic we saw from the DB. I was also asked to investigate the latency to the DB. I noticed that the conversation took 0.3s for a particular example. A large part of this was TCP/IP wait/response times. Previously the DB and all servers were on the same machine. They are now on different machines but that requires TCP/IP. Each query runs in serial, so the TCP/IP overhead is added up 75 times. I have suggested to run these queries in parallel if possible to only suffer the TCP/IP overhead added to the longest query. Feedback welcome. | ||
|
Manit0u
Poland17692 Posts
On June 11 2020 12:23 WarSame wrote: The issue is we were interested in which queries were causing such long replies, so just checking the queries wouldn't work(or would, but you'd have to inspect and reason about each individually). I've done traffic mirroring and put the PCAP into Wireshark. I then analyzed the traffic and found that we had a few reasons for so much traffic. 1. Tons of queries per conversation - ~75 queries to display one page to user. This is not a SPA and is stateless so the whole state must be reloaded every time 2. Queries with extra fields - a number of queries are using select * for tables with ~90 fields. Each field is ~100 bytes, leading to 10 KB replies. When you have 75 replies that's quite a lot. The solutions I've proposed: 1. reduce the number of queries used, use a SPA or something else to reduce it. Long term strategy. 2. don't select *, select the fields you need This should help reduce the traffic we saw from the DB. I was also asked to investigate the latency to the DB. I noticed that the conversation took 0.3s for a particular example. A large part of this was TCP/IP wait/response times. Previously the DB and all servers were on the same machine. They are now on different machines but that requires TCP/IP. Each query runs in serial, so the TCP/IP overhead is added up 75 times. I have suggested to run these queries in parallel if possible to only suffer the TCP/IP overhead added to the longest query. Feedback welcome. Defninitely get rid of stars in your selects. You might also think about adding a cache layer if you fetch the same data often, this would let you avoid the DB queries and round trips. Another thing that comes to mind is also reducing the number of queries. Maybe you can fetch more data with a single query instead of 5? Also, if you have data overlap between the queries you could look into that. | ||
|
WarSame
Canada1950 Posts
They know(and told me when I was starting) that they need to reduce the number of queries but it's a bit of a process. They have a small team so everything takes a while just to work through. I don't think most of their queries are combinable. I think going with SPA might be their best bet for reducing traffic. I know that's on their radar, just going to take a while. | ||
|
Manit0u
Poland17692 Posts
It all depends on how you approach it, at one point we had ~20% of our sprints reserved for stuff like technical debt reduction, code cleanup etc. Sometimes we even did 100% sprint just for that when we had some more leeway (back-end was ahead of front-end and we had to wait for them to catch up). Every company handles this differently but if they could do small incremental changes where each one would help even a little bit with not too much effort I think they should go for it. Like, one developer (not entire team) spending one day of a sprint on removing 1 query from the 75 should not be too disruptive and it would eventually lead to more improvements. | ||
|
WarSame
Canada1950 Posts
You need to keep in mind that they are a company with a small team, not just one team of a larger company. They have 5 people, of which only 2 are heavily technical. Your team, I'd imagine is 7/7 tech people, or maybe with one solution owner. | ||
|
Manit0u
Poland17692 Posts
On June 15 2020 01:26 WarSame wrote: Yeah, I think for them the real answer is to pivot towards more an SPA approach, because that will naturally split up and group their queries, preventing the serial issue. The asterisk issue I think they're willing to spend a significant amount of a sprint or 2 towards which should be all they need. You need to keep in mind that they are a company with a small team, not just one team of a larger company. They have 5 people, of which only 2 are heavily technical. Your team, I'd imagine is 7/7 tech people, or maybe with one solution owner. 7/7 tech. 2 front-end guys and 5 back-end people (including team leader). Another tidbit that could help them with removing asterisks and prepare the queries better for the SPA would be creating the views where you have the data aggregated and you query the view instead of actual tables. They don't have to be materialized views, just planning out and creating the views themselves forces you to get rid of asterisks and allows you to cull the columns you don't need. | ||
|
Manit0u
Poland17692 Posts
This is seriously getting ridiculous. What's next? All the DBs with master/slave replicas and all the OSs with multiple HDDs? | ||
|
Nesserev
Belgium2760 Posts
| ||
| ||
StarCraft 2 StarCraft: Brood War Dota 2 Super Smash Bros Heroes of the Storm Other Games |
|
Sparkling Tuna Cup
RSL Revival
ByuN vs SHIN
Maru vs Krystianer
WardiTV Team League
Patches Events
BSL
Replay Cast
Replay Cast
Wardi Open
Monday Night Weeklies
OSC
[ Show More ] |
|
|
EPT
