|
Thread Rules
1. This is not a "do my homework for me" thread. If you have specific questions, ask, but don't post an assignment or homework problem and expect an exact solution.
2. No recruiting for your cockamamie projects (you won't replace facebook with 3 dudes you found on the internet and $20)
3. If you can't articulate why a language is bad, don't start slinging shit about it. Just remember that nothing is worse than making CSS IE6 compatible.
4. Use [code] tags to format code blocks. |
On August 29 2016 23:18 mantequilla wrote:guys sorry to interrupt your debate about app server threads, but im gonna ask a newb cloud question: As I see Azure has a way to programmatically control cloud resources/deployments ( https://github.com/Azure/azure-sdk-for-java), I'm assuming other cloud providers would have too. Right now have a software that we deploy to cloud by hand (uploading by ftp). We want to sell this web app to numerous small-budget customers. Do you think such a workflow possible/appropriate/viable: We build a website that customers can buy our software through. After payment, using the cloud API's above, an instance of our app is deployed to cloud (creating required services like databases etc) and customer is given a domain name to access his product. All of this is done without any handwork.
Sounds like it would work.
|
On August 29 2016 22:01 Manit0u wrote:Show nested quote +On August 29 2016 20:20 njt7 wrote:On August 26 2016 21:37 tofucake wrote:On August 26 2016 14:31 Wrath wrote:On August 26 2016 06:28 BisuDagger wrote:On August 26 2016 06:20 Wrath wrote:On August 26 2016 05:39 tofucake wrote:
it's single threaded, which is a huge issue Why? Google is a great resource for these types of questions. I know little about node and now im that much more informed after google research! http://stackoverflow.com/questions/17959663/why-is-node-js-single-threaded My question was why it is a huge issue not why it is a single threaded  If you're looking to do some standard website, node is a crap choice. Something like TL would go down in flames if it were made in node, nevermind any big site. Nobody will create the next Facebook/Google/Etsy/Ebay/PayPal/Expedia/Whatever until node gets multithreading. Why would node go down in flames if it ran TL.net? My realworld test gave me about 20 requests/ second on a amazon micro instance. If you really need more just add more instances behind a load balancer. (might have been 50 requests/ second but honestly 20 is enough to prove my point isnt it?). I think that the biggest problem with single-threaded apps is consistency. If each request runs in a separate thread then you can expect the same request to take the same amount of time, not so much with queueing since you don't really control what's being executed when. Another downside of single-threading is that if some process will crash your app it's going to crash it for everyone, not just a single instance.
You never answered my question.
I still see no reason why a non blocking I/O stack couldnt run TL.net. This seems to be a opinionated argument of blocking vs non blocking I/O. As you have said more then once "I dont think javascript should be on the server side". But thats just your opinion and not a reason of why TL.net couldnt be built with node.
In fact I believe a nodejs app would be ideal to build TL.net as it is almost only I/O (writing/ reading) from database. There is no heavy cpu tasks involved that I can see.
|
On August 30 2016 01:20 njt7 wrote:Show nested quote +On August 29 2016 22:01 Manit0u wrote:
I think that the biggest problem with single-threaded apps is consistency. If each request runs in a separate thread then you can expect the same request to take the same amount of time, not so much with queueing since you don't really control what's being executed when.
Another downside of single-threading is that if some process will crash your app it's going to crash it for everyone, not just a single instance. You never answered my question. I still see no reason why a non blocking I/O stack couldnt run TL.net. This seems to be a opinionated argument of blocking vs non blocking I/O. As you have said more then once "I dont think javascript should be on the server side". But thats just your opinion and not a reason of why TL.net couldnt be built with node. In fact I believe a nodejs app would be ideal to build TL.net as it is almost only I/O (writing/ reading) from database. There is no heavy cpu tasks involved that I can see.
I never said TL.net couldn't be built with node. It was someone else.
I will still stand by the opinion that JS has no place in the back-end though. It was designed solely for browser scripting and using it server-side seems like desperately trying to turn a cat into a dog. The fact that you can doesn't mean you should...
On August 29 2016 23:15 obesechicken13 wrote:Hey guys, I have a problem again. I asked it hereShow nested quote +So I'm building a data hub. Basically I want to store large amounts of data in a database.
In order to do that there are these pipes of data that my data providers have to make using my APIs.
I want to secure the APIs so I know who's making the streams and I can limit who makes them. As well as who can send to them. Does it make sense to do OAuth for the non-ingestion API methods and use an API key for the ingestion methods?
OAuth tokens tend to expire and ingestion of data is a long running process.
OAuth + API keys doesn't feel like the right solution since then there's 2 separate security protocols being used.
The other option I see right now is to force users to check the expiration time of their tokens and then try to refresh them if it's about to expire and they still need to send data.
Can't you control the token expiration time? Perhaps even making it semi-infinite (no expiration time but expiring the token when the session is closed)?
|
On August 29 2016 23:18 mantequilla wrote:guys sorry to interrupt your debate about app server threads, but im gonna ask a newb cloud question: As I see Azure has a way to programmatically control cloud resources/deployments ( https://github.com/Azure/azure-sdk-for-java), I'm assuming other cloud providers would have too. Right now have a software that we deploy to cloud by hand (uploading by ftp). We want to sell this web app to numerous small-budget customers. Do you think such a workflow possible/appropriate/viable: We build a website that customers can buy our software through. After payment, using the cloud API's above, an instance of our app is deployed to cloud (creating required services like databases etc) and customer is given a domain name to access his product. All of this is done without any handwork.
Should work. Check out wix as a well-known example of a service that does exactly that.
|
Just discovered Stackless Python and I must say I'm quite impressed. I especially like this benchmarking from one of the documents in there (Introduction to Concurrent Programming With Stackless Python):
Traditional threads
Ten hackysackers going 1000 rounds takes 183 msecs on my computer. Let's increase the number of players
And we get an error when trying to start 10,000 threads on my 3 Ghz machine with 1 Gig of ram. I don't want to bore you with the details of the output, but with some trial and error, the program starts failing at about 1100 threads on my machine. Also note that 1000 threads takes about three times as long as 10.
Stackless
It only takes 19.7 msec. This is almost 10 times faster than the threaded version. Now lets start increasing the number of threadlets
Even by the time we get to 10,000 threads, which the threaded version wasn't even capable of running, we're still running faster than the threaded version did with only 10 threads.
Now I'm trying to keep the code simple, so you'll have to take my word for it, but the increase in timings here is due to the time it takes to setup the hackysack circle. The amount of time to run the game is constant whether you have 10 threadlets or 100000 threadlets. This is because of the way channels work: they block and instantly resume when they get a message. On the other hand, OS threads each take turns checking to see if their message Queue has any elements. This means that the more threads you run, the worse performance gets.
Now I know why EVE used stackless. You can run a million threads with just 100MB of system memory which is great for applications requiring many tiny objects that interact with each other.
|
You've never used green threads before?
|
On August 29 2016 22:01 Manit0u wrote:Show nested quote +On August 29 2016 20:20 njt7 wrote:On August 26 2016 21:37 tofucake wrote:On August 26 2016 14:31 Wrath wrote:On August 26 2016 06:28 BisuDagger wrote:On August 26 2016 06:20 Wrath wrote:On August 26 2016 05:39 tofucake wrote:
it's single threaded, which is a huge issue Why? Google is a great resource for these types of questions. I know little about node and now im that much more informed after google research! http://stackoverflow.com/questions/17959663/why-is-node-js-single-threaded My question was why it is a huge issue not why it is a single threaded If you're looking to do some standard website, node is a crap choice. Something like TL would go down in flames if it were made in node, nevermind any big site. Nobody will create the next Facebook/Google/Etsy/Ebay/PayPal/Expedia/Whatever until node gets multithreading. Why would node go down in flames if it ran TL.net? My realworld test gave me about 20 requests/ second on a amazon micro instance. If you really need more just add more instances behind a load balancer. (might have been 50 requests/ second but honestly 20 is enough to prove my point isnt it?). I think that the biggest problem with single-threaded apps is consistency. If each request runs in a separate thread then you can expect the same request to take the same amount of time, not so much with queueing since you don't really control what's being executed when. Another downside of single-threading is that if some process will crash your app it's going to crash it for everyone, not just a single instance.
On that matter, anyone does some Elixir here ?
I'm willing to get familiar with a backend language that's not fucking PHP and I'm unsure if I should commit more time on Scala or Elixir.
Scala seems to be more popular (well JVM stuff you know, easier to get jobs + the Play framework seems to become quite popular in France), but the BEAM for Elixir just looks awesome. Then there's always Python or even Ruby but I'd like to get familiar with functional programming.
|
On August 30 2016 22:14 Hhanh00 wrote:
You've never used green threads before?
Nope. I never did much multi-threading aside from some personal experiments a long time ago.
On August 30 2016 22:27 -Zoda- wrote:
Scala seems to be more popular (well JVM stuff you know, easier to get jobs + the Play framework seems to become quite popular in France), but the BEAM for Elixir just looks awesome. Then there's always Python or even Ruby but I'd like to get familiar with functional programming.
How about some Erlang then? 
I hear people build awesome stuff with it.
|
On August 30 2016 17:15 Manit0u wrote:Just discovered Stackless Python and I must say I'm quite impressed. I especially like this benchmarking from one of the documents in there ( Introduction to Concurrent Programming With Stackless Python): Traditional threads Show nested quote +
Ten hackysackers going 1000 rounds takes 183 msecs on my computer. Let's increase the number of players
And we get an error when trying to start 10,000 threads on my 3 Ghz machine with 1 Gig of ram. I don't want to bore you with the details of the output, but with some trial and error, the program starts failing at about 1100 threads on my machine. Also note that 1000 threads takes about three times as long as 10.
Stackless Show nested quote +
It only takes 19.7 msec. This is almost 10 times faster than the threaded version. Now lets start increasing the number of threadlets
Even by the time we get to 10,000 threads, which the threaded version wasn't even capable of running, we're still running faster than the threaded version did with only 10 threads.
Now I'm trying to keep the code simple, so you'll have to take my word for it, but the increase in timings here is due to the time it takes to setup the hackysack circle. The amount of time to run the game is constant whether you have 10 threadlets or 100000 threadlets. This is because of the way channels work: they block and instantly resume when they get a message. On the other hand, OS threads each take turns checking to see if their message Queue has any elements. This means that the more threads you run, the worse performance gets.
Now I know why EVE used stackless. You can run a million threads with just 100MB of system memory which is great for applications requiring many tiny objects that interact with each other.
That's really cool. I was thinking of writing a socket-based chat program in python to refresh myself on sockets, and that could be useful. We used pthreads to build a chat program in C with sockets in one of my university classes (it ran like 5 threads at once and was way overkill because my prof was a maniac and thought it would be more challenging to have a bunch of threads to handle each aspect of the chat program). I'd like to try that same design but with python and those stackless microthreads. I bet it'd be a million times less complicated.
|
On August 31 2016 13:28 Ben... wrote:Show nested quote +On August 30 2016 17:15 Manit0u wrote:Just discovered Stackless Python and I must say I'm quite impressed. I especially like this benchmarking from one of the documents in there ( Introduction to Concurrent Programming With Stackless Python): Traditional threads
Ten hackysackers going 1000 rounds takes 183 msecs on my computer. Let's increase the number of players
And we get an error when trying to start 10,000 threads on my 3 Ghz machine with 1 Gig of ram. I don't want to bore you with the details of the output, but with some trial and error, the program starts failing at about 1100 threads on my machine. Also note that 1000 threads takes about three times as long as 10.
Stackless
It only takes 19.7 msec. This is almost 10 times faster than the threaded version. Now lets start increasing the number of threadlets
Even by the time we get to 10,000 threads, which the threaded version wasn't even capable of running, we're still running faster than the threaded version did with only 10 threads.
Now I'm trying to keep the code simple, so you'll have to take my word for it, but the increase in timings here is due to the time it takes to setup the hackysack circle. The amount of time to run the game is constant whether you have 10 threadlets or 100000 threadlets. This is because of the way channels work: they block and instantly resume when they get a message. On the other hand, OS threads each take turns checking to see if their message Queue has any elements. This means that the more threads you run, the worse performance gets.
Now I know why EVE used stackless. You can run a million threads with just 100MB of system memory which is great for applications requiring many tiny objects that interact with each other. That's really cool. I was thinking of writing a socket-based chat program in python to refresh myself on sockets, and that could be useful. We used pthreads to build a chat program in C with sockets in one of my university classes (it ran like 5 threads at once and was way overkill because my prof was a maniac and thought it would be more challenging to have a bunch of threads to handle each aspect of the chat program). I'd like to try that same design but with python and those stackless microthreads. I bet it'd be a million times less complicated.
When I took my operating systems course, the prof made us write a game like space invaders in C using ncurses for graphics. The 'fun' part was that each entity in the game (player, enemies, shots, etc.) had to run in its own thread. It was very much overkill
|
Guys, guys, halp!
I'm working on fixing this legacy web app and there's this action where JS makes an xhr but has to pass through 53723 unique ids in the request. Obviously, setting that as a comma-delimited parameter generates a string that's over 300k characters long. As you might've guessed this is way over any limits for maximum URI length.
Since it's a very fragile, very old, monolithic app I can't really change too much. I was thinking about maybe saving those ids to a temp file somewhere (can JS even do that?) or maybe store them in the session (can you store 300k+ character long strings in the session?) and get them from there in the action but it seems extremely ugly.
How would you approach this? Any suggestions?
|
On September 02 2016 04:20 Manit0u wrote:
Guys, guys, halp!
I'm working on fixing this legacy web app and there's this action where JS makes an xhr but has to pass through 53723 unique ids in the request. Obviously, setting that as a comma-delimited parameter generates a string that's over 300k characters long. As you might've guessed this is way over any limits for maximum URI length.
Since it's a very fragile, very old, monolithic app I can't really change too much. I was thinking about maybe saving those ids to a temp file somewhere (can JS even do that?) or maybe store them in the session (can you store 300k+ character long strings in the session?) and get them from there in the action but it seems extremely ugly.
How would you approach this? Any suggestions?
Is hashing it good enough?
|
Hyrule19265 Posts
On September 02 2016 04:20 Manit0u wrote:
Guys, guys, halp!
I'm working on fixing this legacy web app and there's this action where JS makes an xhr but has to pass through 53723 unique ids in the request. Obviously, setting that as a comma-delimited parameter generates a string that's over 300k characters long. As you might've guessed this is way over any limits for maximum URI length.
Since it's a very fragile, very old, monolithic app I can't really change too much. I was thinking about maybe saving those ids to a temp file somewhere (can JS even do that?) or maybe store them in the session (can you store 300k+ character long strings in the session?) and get them from there in the action but it seems extremely ugly.
How would you approach this? Any suggestions?
post it? can you multipart? or maybe chunk it out?
|
|
|
Sorry guys. I posted this question after 10 hours of doing horrible things to this horrible app. My brain simply shorted out and I completely forgot you could serialize this or something and send it as a POST request Instead I got fixated on various ways you could solve it using GET...
Working with those old, shitty projects is really depressing.
|
So, after 9 months or so of no programming class, and no programming on my own, I am now in the 2nd class for object oriented programming (think of it as a 2nd java class).
And... we kind of jumped in where the last one left off. Problem is I forgot a lot of stuff and am getting confused.
I am wondering if anyone has skype and knows java well and might be willing to help me out a bit so that I can get my bearings again.
|
I studied Java at university, but I don't like the language nowadays. C# is so much better. As any opinion, that is subjective.
Does anyone have any Qt guides? I have used Java's GUI, C# WPF/Win Forms, a little bit of MFC, but I don't know how Qt works. I don't know how I can make my windows automatically readjust themselves when maximised, etc.
|
On September 03 2016 04:21 travis wrote:
So, after 9 months or so of no programming class, and no programming on my own, I am now in the 2nd class for object oriented programming (think of it as a 2nd java class).
And... we kind of jumped in where the last one left off. Problem is I forgot a lot of stuff and am getting confused.
I am wondering if anyone has skype and knows java well and might be willing to help me out a bit so that I can get my bearings again.
What are you confused about?
|
Well, we will see. We have our first project now, and there's been pretty much no review beforehand. Some stuff is coming back to me.. slowly... as I look things up.
It might be a lot of little stuff I don't remember. But if it's gonna turn out to be a "well, we will help you on the forums just post here!" kind of thing I will stay more reserved about the questions I ask and try to look more of it up so I don't spam/annoy the hell out of you all.
edit: it's coming back... passed the first public test on my project. feeeeeeeeeeeeeelin good lol
|
Thanks to all the gods for people like Amit Patel. I've decided to delve a bit deeper into game development and his site has been tremendously helpful so far. Go check it out if you're interested in general math concepts application in games.
Here's some math/science behind hexagons for example: http://www.redblobgames.com/grids/hexagons/
|
|
|
|
|
|
EPT
