|
Thread Rules
1. This is not a "do my homework for me" thread. If you have specific questions, ask, but don't post an assignment or homework problem and expect an exact solution.
2. No recruiting for your cockamamie projects (you won't replace facebook with 3 dudes you found on the internet and $20)
3. If you can't articulate why a language is bad, don't start slinging shit about it. Just remember that nothing is worse than making CSS IE6 compatible.
4. Use [code] tags to format code blocks. |
To anyone working with Doctrine: I've discovered a nasty bug (or maybe it's a feature) today in it.
It appears that if you do something like that...
$queryBuilder = $this->createQueryBuilder();
$expr = $queryBuilder->expr();
$query = $queryBuilder
->where($expr->like($this->getPropertyname('myField'), ':param'))
->setParameter('param', $expr->literal(sprintf('%%s%%', $myVar)))
->getQuery();
$result = $query->getResult();
... the result will be 0 rows. Always.
I've managed to fix it by de-parametrizing it and simply putting the literal expression within the where declaration. I'm still not sure if the error was trying to parametrize the "LIKE" or trying to do literal (which basically wraps your variable into single quotes) as parameter value.
Was a nasty, nasty surprise as this code was running on one of our production servers and our cron which was fetching stuff from another service was creating duplicate entries in the database because of that.
|
Hyrule19217 Posts
Dump the query logs, should tell you what was happening. (please do, we are starting two new projects Symfony now)
|
On January 23 2016 02:47 Manit0u wrote:$query = $queryBuilder
->where($expr->like($this->getPropertyname('myField'), ':param'))
->setParameter('param', $expr->literal(sprintf('%%s%%', $myVar)))
->getQuery();
$result = $query->getResult();
I've managed to fix it by de-parametrizing it and simply putting the literal expression within the where declaration. I'm still not sure if the error was trying to parametrize the "LIKE" or trying to do literal (which basically wraps your variable into single quotes) as parameter value.
Why use literal for a query parameter? Literal is needed if a parameter is not used for some reason. A parameter will be correctly escaped without a literal, and using literal might even break it, producing a query like:
...
WHERE myField LIKE ("'%my_value%'")
Also, the correct print format would presumably be '%%%s%%': https://3v4l.org/MkAhj
|
On January 23 2016 04:30 BByte wrote:Show nested quote +On January 23 2016 02:47 Manit0u wrote:$query = $queryBuilder
->where($expr->like($this->getPropertyname('myField'), ':param'))
->setParameter('param', $expr->literal(sprintf('%%s%%', $myVar)))
->getQuery();
$result = $query->getResult();
I've managed to fix it by de-parametrizing it and simply putting the literal expression within the where declaration. I'm still not sure if the error was trying to parametrize the "LIKE" or trying to do literal (which basically wraps your variable into single quotes) as parameter value. Why use literal for a query parameter? Literal is needed if a parameter is not used for some reason. A parameter will be correctly escaped without a literal, and using literal might even break it, producing a query like: ...
WHERE myField LIKE ("'%my_value%'")
Also, the correct print format would presumably be '%%%s%%': https://3v4l.org/MkAhj
Oh yeah, 3x%. Sorry, was writing code as I went with the post after 17hrs at work... Still, code was fine. I just never knew that literal might have such quirks (and it appears that no one else at the office knew either).
|
Canada16217 Posts
|
On January 23 2016 03:19 tofucake wrote:
Dump the query logs, should tell you what was happening. (please do, we are starting two new projects Symfony now)
Our DB has 100GB traffic every 24h, dumping the logs would be a bad idea.
I'll post some hard-learned lessons later on so that you may avoid several pitfalls we fell into.
|
OK, here go some Symfony tips:
1. Dev env
Make sure to set it up properly. If you're working on Windows you're in a bit of a pickle because when your project grows your dev env is going to suck up your resources (speaking of individual developers' computers) real quick. 2GB memory limit and 300 seconds timeout settings in PHP might not be enough. This is due to Symfony in dev not only rebuilding the cache with each request but also adding additional strain (profiler toolbar, debug etc.), this means over a minute load times on each page. Surprisingly enough, it's not a problem on Linux, must have something to do with WAMP/XAMPP PHP and Apache implementations.
You can of course speed it up by using the xcache (apc for older PHP versions) and autoload dumps + optimizations. Just need to remember about rebuilding them when you add new classes. It's really easy to implement in Symfony since all you have to do is uncomment a few lines in app.php and change one use (XcacheClassLoader instead of ApcClassLoader). And a note of warning, if you're going to use several apps on the same machine you should remember about giving them different cache namespace. Easy way to do it without having to change it every time is simply doing sha1(__FILE__) instead of 'cache_namespace' in app.php. If you won't do that they'll share the namespace and will throw exceptions (cannot redeclare class).
Also, blackfire is your friend.
2. Symfony
Symfony allows you to choose one of the three methods of storing configs: annotation, yaml and xml. Performance-wise there's no difference since in the end it's compiled to pure PHP anyway so just pick one your team is most comfortable with and stick with it (try to not mix it if possible, this leads to confusion). Annotation will mean less files since both yaml and xml are stored separately but it will also be less convenient to find stuff in.
Remember to keep your user object small. It's being reloaded with every request.
Be careful with dependency injection. Especially in event listeners, which are fired with each request so injecting big services into them can hurt you a lot. Thankfully Symfony gives you the option of using lazy injection - learn it, love it.
3. Doctrine
This is the biggest Symfony bottleneck 99% of the time. If you want stuff like Redis you have to think about it from the get-go so that you can make some of your query results go to Redis without having to alter your code later on, when the app grows big and it's a pain in the ass to re-write some repository methods.
Doctrine allows you to cache 3 things: metadata, queries and results. By default it does none of them (even in the prod env!) so it rebuilds metadata cache with each request (this means parsing potentially thousands of files) which slows you down considerably. Just doing simple xcache metadata caching can give you a performance boost up to a factor of 10.
Other things that impact performance are fequent use of flushes and persists when you have to iterate over a large number of tables. Remember to flush AFTER the loop. Disabling Doctrine SqlLogger can also give you a nice boost when you have to do some heavy lifting (we have standard methods that disable the logger before doing heavy stuff and then re-enable it after it's done).
Remember to use proxies.
$post = $this->get->('app.repository.post')->find($postId);
$thread= $this->get->('app.repository.thread')->createNew();
$thread->addPost($post);
$this->entityManager->persist($thread);
$this->entityManager->flush();
This fetches entire objects. You can use proxies to set the relational data without any additional queries.
$post = $this->entityManager->getReference('App\Entity\Post', $postId);
$thread= $this->get->('app.repository.thread')->createNew();
$thread->addPost($post);
$this->entityManager->persist($thread);
$this->entityManager->flush();
Also, make sure to use EXTRA_LAZY fetches on join columns in your configs. This will tell Doctrine to not fetch all of the related objects when you query for one. It'll load them as needed.
Some articles I've found helpful (where you can get more detailed info on what I wrote above):
https://tideways.io/profiler/blog/5-ways-to-optimize-symfony-baseline-performance
http://labs.octivi.com/mastering-symfony2-performance-doctrine/
Hope this helps someone 
|
Hyrule19217 Posts
Might help me out some. We are building on Symfony3/PHP7/nginx on CentOS 6 vagrant boxes. Blackfire hasn't been upgraded to work with 7 yet so we're working without it for now.
|
Looks like Symphony and Zend Framework 2 have about the same problems.
My main gripes with ZF2:
Plugin managers that are horribly inefficient to the point that rendering a small PHP template takes 20ms since every single call to even the simplest of view helpers takes 1-2ms since it goes through the whole plugin manager chain every single time it is called. The same inefficiency is in the service locators, though at least those are called less often and so have less impact. I rewrote half the plugin manager stuff and cut the time by 90%, but I don't understand how that could happen in the first place. Actually, that's wrong, I know how that happened since I've dug in the code enough. They expect that plugins change during run time and sequential calls to the same plugin could have different results - which only makes sense to people that aren't developing actual applications.
The Event Manager is everywhere and it is sloooooow. In addition to the slowness, it also makes everything much harder to debug since you often can't see the root of the problem in the stack traces anymore. Cause the right error in a view and neither the view nor the controller will even appear in the actual stack trace. "There was an error somewhere, fix it".
Billions of includes. We wanted to run the website on the Azure cloud due to our parent company using it. Bad idea. A typical website request ended up in about 450 includes and the distributed source code storage on the cloud meant that every include took about 2ms. That was after using an optimized autoloader that used a full class map instead of looking through include directories. That didn't leave much room for performance optimization. It's much faster on the single linux server it's running on now, but I still can't understand how people manage to manufacture a framework architecture where the framework alone does 400+ includes on a single request. Architecture astronauts are killing the framework, and I thought that was an affliction reserved for AbstractSingletonFactoryPluginLocatorServiceEventManagerHelperJava programmers.
After working with Node.JS for a bit, I'm quite happy to show PHP the door and give it a kick in the back to hurry it's departure. At least for now the architecture astronauts haven't yet put their claws into it. The fact that node.js also loads everything at start time instead of request time is refreshing after trying to optimize ZF2 includes.
Some people are saying 2016 will be the year of simplification, and I say that that can't come soon enough.
Programmers need to focus on what their programs and frameworks need to do, not on what it could be used for. It's nice when the framework is flexible, but in the end it's still a web framework that is supposed to do web stuff, not guide a rocket to space. It's good when a Framework can't do everything, but instead does one job very well.
|
|
|
What that is essentially saying is: Cache everything with Varnish and then the poor performance of Symphony doesn't matter anymore, because you are only serving stuff from cache. Oh, and you must not use the slow parts of Symphony like ORM and your application must be a REST service and not actually serve any templated content.
I have no doubt that you can tweak every framework until you eventually have something fast, but for some frameworks the performance tuning takes up much more time than using the framework would have saved in the first place.
PHP as a language itself is not as bad as it used to be and performance wise it's fine, just some of the major frameworks are terrible.
|
True enough. But even without all this caching and with very little optimization (and still using ORM and templating) we can easily get to 500ms response times in our apps. That's sufficient most of the time. 30ms is great for REST stuff but with actual views it's nigh-unachievable due to rendering times and amount of data you have to send.
|
The Octivi article may be an extreme example, but it's certainly possible to get decent performance with Symfony. We get response times of roughly 80 ms on simple pages, but that still includes several DB queries and Twig templates -- not that the latter really matters, templates get cached to PHP code anyway. That's on PHP 5.6 and even if PHP 7 isn't 100 % faster, it's still going to make a noticeable difference.
This has required no particular optimizations, in fact our service count sometimes terrifies me. But opcode cache is essential for any PHP application and Doctrine metadata cache is pretty much required as well. The latter should probably be expressed more clearly in Symfony docs as well, even if the cache can't be enabled by default because it requires an external caching solution.
More than 95 % of the time any performance problems relate the database anyway, and at that point the language and the framework only matter if they get in your way too much. For example using references with Doctrine is likely a micro optimization. Fetching a single record using the primary key typically isn't the performance problem. Of course doing it inside a loop can be an issue and sometimes references may be the right tool for the job. Same goes for the extra lazy collections -- sometimes useful, rarely needed. Overall query count is naturally important, and Doctrine can both help and hinder here. Sometimes collections and nested structures can make it hard to see why there are for example multiple queries. On the other hand, debugging is usually easy.
I'm not a huge fan of Doctrine overall and some Symfony components (Form and PropertyAccess come to mind) have problems as well. While the modern PHP frameworks were probably mainly influenced by Rails, at least Symfony certainly has Java roots as well. That means it's certainly not for everyone. But I haven't seen the ZF2 problems like bad exception tracking.
Overall I like writing JavaScript (ES6) a lot. For small projects it's great and for larger projects Flow or TypeScript can help. The ecosystem seems like a mess though. Not locking vendor libraries to a specific version (by default) is a bother and the tool stacks have so many dependencies it's no wonder that minor semver violations cause huge compatibility problems.
(And if a framework were to do only one thing, it wouldn't be a framework, it would be a library.)
|
|
|
|
|
You guys have been really helpful in this thread, I'm reading some cool stuff about web development that is way over my head. Here's me trying to learn some javscript, trying to make a queue, of course it's not working out for me:
function queue(arr, item) {
testArr.shift(arr);
testArr.push(item);
return item; // just to see if anything works
}
queue([], 1); //doesn't work even if [] is replaced with a number.
var testArr = [1,2,3,4,5];
I'm trying to remove the front item of the array, then add the second number when the queue(); function is called.
I'm getting a TypeError: Cannot read property 'shift' undefined.
It works fine if I substitute in some numbers for the arguments I'm trying to pass.
|
|
|
Hmm ok. Ill read that chapter and come back if I need more help.
It works fine if I simply remove the arguments of the method and just substitute in numbers, so the problem must be wrongheaded thinking on the shift and push.
Thanks.
|
ugh... moved to vs2015 at work last week, and it takes siginificantly more memory to link... and since we statically link LIKE FUCKING EVERYTHING in this place linking some of our exes takes over 15 minutes now, consuming 5.5GB of memory in the process. Oh well, at least it gets people talking about using DLLs for some of our shit...
|
On January 28 2016 10:14 Cyx. wrote:ugh... moved to vs2015 at work last week, and it takes siginificantly more memory to link... and since we statically link LIKE FUCKING EVERYTHING in this place linking some of our exes takes over 15 minutes now, consuming 5.5GB of memory in the process. Oh well, at least it gets people talking about using DLLs for some of our shit...
Is it Visual Studio really? What more is it doing then?
|
|
|
|
|
|
EPT![[image loading]](https://raw.githubusercontent.com/nvbn/thefuck/master/example.gif)
