|
Thread Rules 1. This is not a "do my homework for me" thread. If you have specific questions, ask, but don't post an assignment or homework problem and expect an exact solution. 2. No recruiting for your cockamamie projects (you won't replace facebook with 3 dudes you found on the internet and $20) 3. If you can't articulate why a language is bad, don't start slinging shit about it. Just remember that nothing is worse than making CSS IE6 compatible. 4. Use [code] tags to format code blocks. |
Hi,
Can someone help me with some Java. I read HashMap have a race condition where if you concurrently modify their size you can end in an infinite loop. I'm trying to get a POC code that might trigger it, but so far I have no luck being able to replicate it. Obviously being a race condition I guess I can't get it 100%, I'd just like to see I'm on the right track.
Basically I create one static HashMap and then create an executor service and submit a ton of jobs that either add or update a key in the hashmap (some adds to resize map).
Any ideas?
edit: nm - used this code - http://java.dzone.com/articles/java-7-hashmap-vs, able to replicate. need to figure out why my code was wrong...
edit: hmm what I changed in my code was instead of submitting 500k tasks that all add/update a key, I submit 4 tasks each of which try to submit/update 500k items. Attempted code from the tutorial into this and it loops, thread dump shows its stuck on hashmap stuff, and replacing it with concurrent hash map makes it work.
code + Show Spoiler + import java.util.HashMap; import java.util.Map; import java.util.concurrent.CountDownLatch; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors;
public class Executor { static final int NUM_TESTS = 3; static final int NUM_THREAD = 4; static final int THREAD_ITERATIONS = 500000; //500k static final Map<String, Integer> map = new HashMap<>(1);
public static void main(String[] args) throws InterruptedException { for (int test = 0; test < NUM_TESTS; test++) { System.out.println("Started at: " + System.currentTimeMillis()); ExecutorService es = Executors.newFixedThreadPool(NUM_THREAD); final CountDownLatch latch = new CountDownLatch(NUM_THREAD); for (int i = 0; i < NUM_THREAD; i++) { es.execute(() -> { for (int k = 0; k < THREAD_ITERATIONS; k++) { Integer newInteger1 = (int) Math.ceil(Math.random() * 1000000); Integer newInteger2 = (int) Math.ceil(Math.random() * 1000000);
// 1. Attempt to retrieve a random Integer element Integer retrievedInteger = map.get(String.valueOf(newInteger1));
// 2. Attempt to insert a random Integer element map.put(String.valueOf(newInteger2), newInteger2); } latch.countDown(); }); } es.shutdown(); latch.await(); System.out.println(map.size()); System.out.println("Finished at: " + System.currentTimeMillis()); } } }
|
You do know that if you have multiple threads accessing the same data structure concurrently is going to have race conditions right?
ConcurrentHashMap/ConcurrentAnything have mutex locks (probably some sort of monitor) on their state-modifying member functions, so that for example, two calls to Add() at the same time don't fuck up the data structure. I'm guessing you haven't done much parallism/threading stuff? Parallelism isn't just achieved by spinning worker threads. In most cases you'll have the same performance with just one thread if you don't actually have a good reason to apply concurrency optimizations.
|
On March 07 2015 18:15 Blisse wrote: You do know that if you have multiple threads accessing the same data structure concurrently is going to have race conditions right?
ConcurrentHashMap/ConcurrentAnything have mutex locks (probably some sort of monitor) on their state-modifying member functions, so that for example, two calls to Add() at the same time don't fuck up the data structure. I'm guessing you haven't done much parallism/threading stuff? Parallelism isn't just achieved by spinning worker threads. In most cases you'll have the same performance with just one thread if you don't actually have a good reason to apply concurrency optimizations.
Lol idk you read my thread at all. I know theres race condition with infinite loop, I just wanted to reliably replicate it. In practice i see it in work j2ee code basic that basically ends up with static maps wrapped in beans or cdi shit. I was debugging the issue in prod and coworkers are being annoying about understanding need to use concurrent maps so i wanted poc to show them.
I know i can use concurrent map to fix it, its what i freaking said in my post (concurrenthashmap will fix it). now i can just convince stubborn coworkers why they can just use concurrenthashmap and not complain.
|
On March 07 2015 18:25 teamamerica wrote:Show nested quote +On March 07 2015 18:15 Blisse wrote: You do know that if you have multiple threads accessing the same data structure concurrently is going to have race conditions right?
ConcurrentHashMap/ConcurrentAnything have mutex locks (probably some sort of monitor) on their state-modifying member functions, so that for example, two calls to Add() at the same time don't fuck up the data structure. I'm guessing you haven't done much parallism/threading stuff? Parallelism isn't just achieved by spinning worker threads. In most cases you'll have the same performance with just one thread if you don't actually have a good reason to apply concurrency optimizations. Lol idk you read my thread at all. I know theres race condition with infinite loop, I just wanted to reliably replicate it. In practice i see it in work j2ee code basic that basically ends up with static maps wrapped in beans or cdi shit. I was debugging the issue in prod and coworkers are being annoying about understanding need to use concurrent maps so i wanted poc to show them. I know i can use concurrent map to fix it, its what i freaking said in my post (concurrenthashmap will fix it). now i can just convince stubborn coworkers why they can just use concurrenthashmap and not complain.
edit: besides im not just talking about generic race condition like getting the wrong data in a get - there is specific to hashmap, infinite loop race condition. which again i mentioned something specifically.
im so confused as to who youre talking to in your comment because its sure as hell not even responding to almost anything i asked...
|
On March 07 2015 10:20 darkness wrote:
Anyway, if it's the HP field that you want to access, then make it a protected field in the base class. If this isn't what you want, you should indeed reconsider design.
What base class? The component class? No no no no no.
The main point of component based entity systems is to eliminate massive inheritance hierarchies and a bajillion different entity types that would result from doing things like you suggest.
Component based design gives you just one entity class, hides the necessary evil of type casting neatly behind its own interface, and unifies the treatment of components and systems. Using a type hiding class can mean that components don't even have to be aware of the Component base class. As long as a class implements the correct static interface, it can act as a component.
I'm not sure there's a neater way to abstract this in C++.
|
On March 07 2015 18:25 teamamerica wrote:Show nested quote +On March 07 2015 18:15 Blisse wrote: You do know that if you have multiple threads accessing the same data structure concurrently is going to have race conditions right?
ConcurrentHashMap/ConcurrentAnything have mutex locks (probably some sort of monitor) on their state-modifying member functions, so that for example, two calls to Add() at the same time don't fuck up the data structure. I'm guessing you haven't done much parallism/threading stuff? Parallelism isn't just achieved by spinning worker threads. In most cases you'll have the same performance with just one thread if you don't actually have a good reason to apply concurrency optimizations. Lol idk you read my thread at all. I know theres race condition with infinite loop, I just wanted to reliably replicate it. In practice i see it in work j2ee code basic that basically ends up with static maps wrapped in beans or cdi shit. I was debugging the issue in prod and coworkers are being annoying about understanding need to use concurrent maps so i wanted poc to show them. I know i can use concurrent map to fix it, its what i freaking said in my post (concurrenthashmap will fix it). now i can just convince stubborn coworkers why they can just use concurrenthashmap and not complain.
lol wow you get angry easily. forgive me if i misinterpret a confusing post where your goal is to reproduce a race condition. remind me next time to not read your posts if this is how you talk to people who misread you. no wonder your coworkers are having a hard time if you act like this..
|
|
On March 08 2015 07:41 Blisse wrote:Show nested quote +On March 07 2015 18:25 teamamerica wrote:On March 07 2015 18:15 Blisse wrote: You do know that if you have multiple threads accessing the same data structure concurrently is going to have race conditions right?
ConcurrentHashMap/ConcurrentAnything have mutex locks (probably some sort of monitor) on their state-modifying member functions, so that for example, two calls to Add() at the same time don't fuck up the data structure. I'm guessing you haven't done much parallism/threading stuff? Parallelism isn't just achieved by spinning worker threads. In most cases you'll have the same performance with just one thread if you don't actually have a good reason to apply concurrency optimizations. Lol idk you read my thread at all. I know theres race condition with infinite loop, I just wanted to reliably replicate it. In practice i see it in work j2ee code basic that basically ends up with static maps wrapped in beans or cdi shit. I was debugging the issue in prod and coworkers are being annoying about understanding need to use concurrent maps so i wanted poc to show them. I know i can use concurrent map to fix it, its what i freaking said in my post (concurrenthashmap will fix it). now i can just convince stubborn coworkers why they can just use concurrenthashmap and not complain. lol wow you get angry easily. forgive me if i misinterpret a confusing post where your goal is to reproduce a race condition. remind me next time to not read your posts if this is how you talk to people who misread you. no wonder your coworkers are having a hard time if you act like this..
for u blisse <3 + Show Spoiler + lol you give me a condescending ass answer of " I'm guessing you haven't done much parallism/threading stuff? Parallelism isn't just achieved by spinning worker threads. In most cases you'll have the same performance with just one thread if you don't actually have a good reason to apply concurrency optimizations.".
My post says in the first line: "Can someone help me with some Java. I read HashMap have a race condition where if you concurrently modify their size you can end in an infinite loop. I'm trying to get a POC code that might trigger it, but so far I have no luck being able to replicate i"
what is confusing about this? I CLEARLY say I'm trying to replicate this race condition in the first sentence. did you just read a word in my post? What is your whole post in response to? Please tell me how you interpreted my post that you decided you'd just give me a lecture of generic programming platitudes. "lol wow you get defensive easily" - see, I can say the same thing about you.
also thx for judging me so much, that's great. my coworkers treat me with respect so I treat them with respect. you treat me like a 5 yr old in your answer, i will treat you same way if i feel like it.
anyway, sorry for the last 3 posts by me being worthless. tldr guys - use concurrenthashmap (you can synchronize it externally but concurrent probably faster) if you are concurrently modifying a hashmap. beyond this easily noticeable condition, theres tons of other stuff most likely that might go wrong. concurrency hard ^^
this condition is nice b/c at least its hard not to notice your JVM suddenly eating all your CPU and stuck threads clearly stands out in thread dumps, vs just silent corruption. the cause of condition has to do with hashmap, resizing, and chaining used to resolve hash collisions. basically when resizing hashmap it will iterate through chains to plop nodes in new bucket, but if two threads hit resize, when its iterating through a chain node => node.next => node.next might end up back at original node (i.e. now we're in infinite loop).
|
My personal experience with non-concurrent hashmaps modified concurrently is that it often crashes with an exception when you iterate on them, but I didn't imagine it could even loop like that :D So yeah, thank you, ConcurrentHashMap...
|
On March 08 2015 08:38 teamamerica wrote:Show nested quote +On March 08 2015 07:41 Blisse wrote:On March 07 2015 18:25 teamamerica wrote:On March 07 2015 18:15 Blisse wrote: You do know that if you have multiple threads accessing the same data structure concurrently is going to have race conditions right?
ConcurrentHashMap/ConcurrentAnything have mutex locks (probably some sort of monitor) on their state-modifying member functions, so that for example, two calls to Add() at the same time don't fuck up the data structure. I'm guessing you haven't done much parallism/threading stuff? Parallelism isn't just achieved by spinning worker threads. In most cases you'll have the same performance with just one thread if you don't actually have a good reason to apply concurrency optimizations. Lol idk you read my thread at all. I know theres race condition with infinite loop, I just wanted to reliably replicate it. In practice i see it in work j2ee code basic that basically ends up with static maps wrapped in beans or cdi shit. I was debugging the issue in prod and coworkers are being annoying about understanding need to use concurrent maps so i wanted poc to show them. I know i can use concurrent map to fix it, its what i freaking said in my post (concurrenthashmap will fix it). now i can just convince stubborn coworkers why they can just use concurrenthashmap and not complain. lol wow you get angry easily. forgive me if i misinterpret a confusing post where your goal is to reproduce a race condition. remind me next time to not read your posts if this is how you talk to people who misread you. no wonder your coworkers are having a hard time if you act like this.. for u blisse <3 + Show Spoiler + lol you give me a condescending ass answer of " I'm guessing you haven't done much parallism/threading stuff? Parallelism isn't just achieved by spinning worker threads. In most cases you'll have the same performance with just one thread if you don't actually have a good reason to apply concurrency optimizations.".
My post says in the first line: "Can someone help me with some Java. I read HashMap have a race condition where if you concurrently modify their size you can end in an infinite loop. I'm trying to get a POC code that might trigger it, but so far I have no luck being able to replicate i"
what is confusing about this? I CLEARLY say I'm trying to replicate this race condition in the first sentence. did you just read a word in my post? What is your whole post in response to? Please tell me how you interpreted my post that you decided you'd just give me a lecture of generic programming platitudes. "lol wow you get defensive easily" - see, I can say the same thing about you.
also thx for judging me so much, that's great. my coworkers treat me with respect so I treat them with respect. you treat me like a 5 yr old in your answer, i will treat you same way if i feel like it.
anyway, sorry for the last 3 posts by me being worthless. tldr guys - use concurrenthashmap (you can synchronize it externally but concurrent probably faster) if you are concurrently modifying a hashmap. beyond this easily noticeable condition, theres tons of other stuff most likely that might go wrong. concurrency hard ^^ this condition is nice b/c at least its hard not to notice your JVM suddenly eating all your CPU and stuck threads clearly stands out in thread dumps, vs just silent corruption. the cause of condition has to do with hashmap, resizing, and chaining used to resolve hash collisions. basically when resizing hashmap it will iterate through chains to plop nodes in new bucket, but if two threads hit resize, when its iterating through a chain node => node.next => node.next might end up back at original node (i.e. now we're in infinite loop).
i treated you like a 5 year old because i misinterpreted your post and i have no clue who you are over the internet - experienced or new. i assumed new because i read (misinterpreted at this point) you having obvious threading problems. however the way you're responding now shows the type of person you are, and i'm sorry you feel that i've insulted you initially (you deserved my second comment imo for your replies there), but really you've blown this ridiculously out of proportion and i honestly hope you're capable of apologizing sincerely at this point for continuing this as far as you have thus far and stopping your nonsense.
|
On March 08 2015 11:40 Blisse wrote: i treated you like a 5 year old because i misinterpreted your post and i have no clue who you are over the internet - experienced or new. i assumed new because i read (misinterpreted at this point) you having obvious threading problems. however the way you're responding now shows the type of person you are, and i'm sorry you feel that i've insulted you initially (you deserved my second comment imo for your replies there), but really you've blown this ridiculously out of proportion and i honestly hope you're capable of apologizing sincerely at this point for continuing this as far as you have thus far and stopping your nonsense. Not that I really want to get into the personal argument on the internet, but I don't really think you're definitively the good guy in this discussion...
|
+ Show Spoiler +I'll apologize for trying to get on his nerves by saying he doesn't get along with his teammates but not unless he apologizes for needlessly cursing at me and replying in anger. This could have ended immediately if he just replied "I know what ConcurrentHashMaps are, I'm just trying to reproduce a race condition", but he didn't, and I don't see anything wrong with my initial reply since I was asking questions and making assumptions. I'm sorry there are miscommunications on the Internet but if you feel the need to swear at people because of that, then well I'd expect an apology because there was absolutely no need for that.
edit: actually I read over his two responses and it wasn't that bad so yeah i totally went overboard too. i've started to skim over stuff once i read swearing at me.
sorry about that teamamerica, i'll really have to make sure i don't respond to posts at 3am again >_>
|
Hyrule18982 Posts
everybody shut the hell up :D
|
microsofts c++ compiler is a god damned idiot. it kept crashing without telling me why, yet having the nerve to suggest i ought to simplify code that was completely unrelated to the crash.
what is the point of this component / entity based design? i get a consistent handle to something for the price of having all my types inherit from some useless baseclass? i can get a consitent handle for something wihtout paying that price. i'm not sure i understand it at all.
and if you are storing stuff via pointer, sure the owners, of the pointer are stacked up nicely in your blob of memory but you have no guarantee that the objects they own are. so when you iterate over the collection, you are still jumping all over the place in memory.
|
Shit, I always read your posts like poetry, it's pretty priceless xD. Everything you say sounds so much deeper.
|
i'm shooting for nice rectangular shapes is all.
|
On March 08 2015 15:30 nunez wrote: microsofts c++ compiler is a god damned idiot. it kept crashing without telling me why, yet having the nerve to suggest i ought to simplify code that was completely unrelated to the crash.
what is the point of this component / entity based design? i get a consistent handle to something for the price of having all my types inherit from some useless baseclass? i can get a consitent handle for something wihtout paying that price. i'm not sure i understand it at all.
and if you are storing stuff via pointer, sure the owners, of the pointer are stacked up nicely in your blob of memory but you have no guarantee that the objects they own are. so when you iterate over the collection, you are still jumping all over the place in memory. I think performance wise it "should" be better because you can split most actions up so that you can make most of your program run on cache instead of ram, but im not really knowledgeable about performance.
|
On March 08 2015 19:38 sabas123 wrote:Show nested quote +On March 08 2015 15:30 nunez wrote: microsofts c++ compiler is a god damned idiot. it kept crashing without telling me why, yet having the nerve to suggest i ought to simplify code that was completely unrelated to the crash.
what is the point of this component / entity based design? i get a consistent handle to something for the price of having all my types inherit from some useless baseclass? i can get a consitent handle for something wihtout paying that price. i'm not sure i understand it at all.
and if you are storing stuff via pointer, sure the owners, of the pointer are stacked up nicely in your blob of memory but you have no guarantee that the objects they own are. so when you iterate over the collection, you are still jumping all over the place in memory. I think performance wise it "should" be better because you can split most actions up so that you can make most of your program run on cache instead of ram, but im not really knowledgeable about performance. I don't even know what you're talking about with that splitting actions up part, but I do know that you are thinking about performance without any need to do so. Also if my understanding of entity/component design is correct and a quick google search is to be trusted, it is rather bad for performance than good because every time you are trying to do something, you have to filter a list of random stuff for the stuff that you can work on before you can do that work. But once again, that's a pointless consideration. Messing up your static typing (which you should be really glad to have) by type casting is much more problematic.
|
@sabas be better than what?
to specify, i was commenting nerthert post and example, which i might be misunderstanding,. he holds pointers in vectors, and not the actual objects. thus the objects are not layed out in continguous memory, and you memory access pattern is at the mercy of your allocator.
|
On March 08 2015 19:56 spinesheath wrote:Show nested quote +On March 08 2015 19:38 sabas123 wrote:On March 08 2015 15:30 nunez wrote: microsofts c++ compiler is a god damned idiot. it kept crashing without telling me why, yet having the nerve to suggest i ought to simplify code that was completely unrelated to the crash.
what is the point of this component / entity based design? i get a consistent handle to something for the price of having all my types inherit from some useless baseclass? i can get a consitent handle for something wihtout paying that price. i'm not sure i understand it at all.
and if you are storing stuff via pointer, sure the owners, of the pointer are stacked up nicely in your blob of memory but you have no guarantee that the objects they own are. so when you iterate over the collection, you are still jumping all over the place in memory. I think performance wise it "should" be better because you can split most actions up so that you can make most of your program run on cache instead of ram, but im not really knowledgeable about performance. I don't even know what you're talking about with that splitting actions up part, but I do know that you are thinking about performance without any need to do so. Also if my understanding of entity/component design is correct and a quick google search is to be trusted, it is rather bad for performance than good because every time you are trying to do something, you have to filter a list of random stuff for the stuff that you can work on before you can do that work. But once again, that's a pointless consideration. Messing up your static typing (which you should be really glad to have) by type casting is much more problematic. My idea was to have some kind of sorter that will do the filter and then give that to the function/system/processor (how ever you want to call it).
My intention isn't performance, but from what I read it should be possible to have faster performance with this kind of architecture then the traditional class hierarchy
|
|
|
|