EPTI believe that this is regarded as one of the best. It's also hard
. (at least for me)The Big Programming Thread - Page 1003
| Forum Index > General Forum |
Thread Rules 1. This is not a "do my homework for me" thread. If you have specific questions, ask, but don't post an assignment or homework problem and expect an exact solution. 2. No recruiting for your cockamamie projects (you won't replace facebook with 3 dudes you found on the internet and $20) 3. If you can't articulate why a language is bad, don't start slinging shit about it. Just remember that nothing is worse than making CSS IE6 compatible. 4. Use [code] tags to format code blocks. | ||
|
Deleted User 3420
24492 Posts
I believe that this is regarded as one of the best. It's also hard . (at least for me) | ||
|
Manit0u
Poland17845 Posts
Get assigned to a Scala project. Remove some code. Project now works fine on 2 machines instead of 8 with the same load. Feel good. | ||
|
Deleted User 3420
24492 Posts
I have used something like that before actually (not that specifically, that one is for looking at cryptocurrency stuff? I don't know about any of that stuff so if what I just said was dumb then forgive me). Anyways it was very, very interesting to see some of the intuition I had about some of the graphs I was working with come to life in a visual way. However, the really hard graphs were difficult to even tell what was going on when their relationships were inspected visually. They mostly come out looking like repeated intertwined rings of varying lengths. | ||
|
Manit0u
Poland17845 Posts
On March 15 2019 01:40 travis wrote: Manit0u I just realized I never replied to your post about that graph visualizer. I have used something like that before actually (not that specifically, that one is for looking at cryptocurrency stuff? I don't know about any of that stuff so if what I just said was dumb then forgive me). Anyways it was very, very interesting to see some of the intuition I had about some of the graphs I was working with come to life in a visual way. However, the really hard graphs were difficult to even tell what was going on when their relationships were inspected visually. They mostly come out looking like repeated intertwined rings of varying lengths. Well, those visualizations are actually for function calls and network systems. Could potentially work for anything since functions within an application and network systems are basically graphs of sort. | ||
|
WarSame
Canada1950 Posts
| ||
|
Mr. Wiggles
Canada5894 Posts
On March 14 2019 03:34 SC-Shield wrote: Could you please recommend a few nice books about Machine Learning? If they're about Reinforcement Learning, then it will be even better. Here's the book I used when I took a course in reinforcement learning from Rich Sutton: http://incompleteideas.net/book/the-book.html It's available for free as a PDF. When we did the course he was still working on the second edition, so some stuff was missing, but it looks complete now. It was pretty good from what I remember, and was useful when I was refreshing myself on some concepts recently. Rich is one of the 'fathers' of reinforcement learning and is currently leading the Deep Mind office in Edmonton, so you can consider him a pretty authoritative source on RL  https://deepmind.com/blog/deepmind-office-canada-edmonton/ | ||
|
Manit0u
Poland17845 Posts
![[image loading]](https://i.imgur.com/Ncp9FiD.png) | ||
|
Manit0u
Poland17845 Posts
Unfortunately I haven't touched front-end for 3 years and I wouldn't know what would be some of the best online resources to learn those technologies. Could you hook the brother up? Any tips or hints on what to pay attention to and what extra skills (besides less or sass) might be required would be greatly appreciated. Should I also teach him some about NoSQL stuff like Mongo? | ||
|
Deleted User 3420
24492 Posts
So, it's my intuition that what a neural network really does is basically find and weight correlations between (n choose k) inputs within the units of the neural network. Which makes me wonder: for many networks, is relu really a good choice of activation function? Because, if the above is correct, then doesn't relu only find POSITIVE correlations between inputs? For example, if we have inputs A,B,C,D, and are trying to classify a cat, a network using relu can find that A = .5 AND B = .5, may be more significant towards our data being a cat than the individual weightings of when A = .5 + B = .5. However, relu should *not* be able to find a negative correlation, that is to say that maybe A = .5 increases the likelihood of our data being a cat, C = .5 increases the likelihood of our data being a cat, but A =.5 AND C = .5 makes it LESS likely that our data is a cat. Am I right that relu cannot find that last type of correlation, and you would need something like leaky relu to find that kind of correlation in data? | ||
|
Equalizer
Canada115 Posts
| ||
|
Deleted User 3420
24492 Posts
Let's take a network with 26 inputs: A....Z If 100% of the time that A >= .5, and C >=.5: it is not a cat. But 100% of the rest of the time A >= .5, or C >=.5: it is a cat It's not realistically ever going to be able to learn this rule exactly, right? Even if some later unit doesn't fire because it solved {A >= .5, C>=.5} ---> not a cat, this won't prevent it from thinking it is a cat based on the other inputs. I mean.. I suppose I could see it eventually solving that relationship. But then the amount of layers and units would need to be incredibly huge. Or am I overcomplicating this and I am flat out wrong in my conceptualization. | ||
|
Simberto
Germany12004 Posts
a correlation of -1 between A and B means that if A, then never B (and if B, then never A) a correlation of 0 means that A doesn't influence whether B or not B a correlation of 1 means that if A, then B (and if B, then A) all other values are in between. I don't know too much about programming though. | ||
|
Acrofales
Spain18445 Posts
On March 18 2019 02:29 travis wrote: But in the end, it can never value the correlation as negative, at best it can value it at zero, right? So it can't fully capture the relationship in the most accurate way - your network can't truly punish this negative correlation - it can only "not reward it". Let's take a network with 26 inputs: A....Z If 100% of the time that A >= .5, and C >=.5: it is not a cat. But 100% of the rest of the time A >= .5, or C >=.5: it is a cat It's not realistically ever going to be able to learn this rule exactly, right? Even if some later unit doesn't fire because it solved {A >= .5, C>=.5} ---> not a cat, this won't prevent it from thinking it is a cat based on the other inputs. I mean.. I suppose I could see it eventually solving that relationship. But then the amount of layers and units would need to be incredibly huge. Or am I overcomplicating this and I am flat out wrong in my conceptualization. You just need multiple layers as Equalizer stated. I haven't actually read this blog, but the solution appears to be right, so I assume it's on-point for solving XOR with perceptrons: https://towardsdatascience.com/perceptrons-logical-functions-and-the-xor-problem-37ca5025790a | ||
zatic
Zurich15366 Posts
travis in the end it comes down to what Equalizer said, a negative weight on the edge to the next layer does the job. And it doesn't need a lot of layers and units. Maybe just try it out? You can build your problem (essentially XOR) with 2 layers, (in and output layer), 2 nodes, and fit it to 1. accuracy. If you print out the weights and biases you will see that one edge is positive and one is negative. Conceptually, it's not the activation function that does the regression. You can try above with any activation function, including one that can return negative values, and it will turn out the same. | ||
|
Deleted User 3420
24492 Posts
Anyways I will just take everyone's word for it that relu works efficiently for this. I do know that most papers say that evidence points at there not being much advantage to leaky relu other than for addressing the issue of units that will no longer learn. | ||
|
zatic
Zurich15366 Posts
| ||
|
Equalizer
Canada115 Posts
Consider the following, output = w_1 * A + w_2 * B + w_3 * max(A + B - 1,0) + bias where A,B in [0,1] By setting w_3 as an appropriate negative weight you can apply any negative contribution from the event of A and B to the output that is needed. Note: max(A + B - 1,0) is just ReLu with weights of 1 and bias of -1. | ||
|
Deleted User 3420
24492 Posts
| ||
|
Manit0u
Poland17845 Posts
| ||
|
JimmyJRaynor
Canada17866 Posts
On February 21 2019 12:21 Manit0u wrote: Screw math and ML. to your point... In general, "Machine Learning" is being oversold by its proponents. "I apparently have a bit of a reputation as someone who is anti-machine learning or anti-AI when it comes to human research. This is a bit of misrepresentation of my views, and (I'd argue) a misrepresentation of the issues "statistics people" take with AI/ML as a whole." "I personally think that AI/ML has a lot to bring to the table to enhance science, health and human performance. The problem is that the AI/ML crowd are over-selling their wares and often being disingenuous about what is current state-of-the art" "Issue 1: CLAIMING EVERYTHING IS MACHINE LEARNING. Just because AI/ML may use algebra or linear regression, doesn't mean it is AI/ML. Same goes for Nonlinear regression, correlation, logistic regression, or everything else that IS STATISTICS (or information theory, etc.) " "It's cool if you use statistics and statistical concepts properly. Really, we're a big-tent kind of people. Just don't claim you invented something you clearly did not. And no, stringing together multiple correlations in an automated way doesn't make it extra special. " "Issue 2: OMG THE HYPE MACHINE, MAKE IT STOP. Again, a lot of really good stuff is being trialed with AI/ML. You don't need to oversell the genuinely good work and advances being pioneered. Here's the thing, most "statistics people" are allergic to hype." "Many human research statisticians work in areas of health where people can die or receive in appropriate treatments if we do our job wrong. It isn't to say we're perfect, but we work hard to be conservative and criticize our models so we're confident in the results." "This is, I think, the main reason statisticians have issues with the AI/ML crowd: we can smell snake oil. The really good and avant garde AI/ML work gets lumped in with the utter nonsense directed at VC's and the pop media." "Issue 3: THE SNAKE OIL IS SPREADING (tweet #7) Again, there is good AI/ML work being done, but most of it is just re-branded statistics or 'stuff' hiding behind the term "proprietary". This snake oil is leaking into government, academia, etc in an attempt to be 'cool'" "We're seeing this salesmanship more-and-more outside of traditional AI/ML technology circles. There are conference presentations or academic papers that call things like principle component analysis AI/ML... it was invented in 1901! https://en.wikipedia.org/wiki/Principal_component_analysis … " "Finally, many "stats people" are interested in applying AI/ML techniques and seeing where it can compliment our backgrounds and current work. We just get turned off by the bravado, hype and salesmanship that accompanies AI/ML. So yeah, we'll keep giving it a hard time. " | ||
| ||
