The Big Programming Thread - Page 795

Prev 1 793 794 795 796 797 1032 Next

Thread Rules
1. This is not a "do my homework for me" thread. If you have specific questions, ask, but don't post an assignment or homework problem and expect an exact solution.
2. No recruiting for your cockamamie projects (you won't replace facebook with 3 dudes you found on the internet and $20)
3. If you can't articulate why a language is bad, don't start slinging shit about it. Just remember that nothing is worse than making CSS IE6 compatible.
4. Use [code] tags to format code blocks.

Manit0u

Poland17420 Posts

November 09 2016 14:00 GMT

#15881

On November 09 2016 22:15 Djagulingu wrote:
Dear Django,

Can you be less opinionated pls? Kthx.

Sincerely.

P.S: I still like you.

Django being opinionated isn't such big of a problem (depending on what you want to do). Single request generating 1.6 million db queries is (something I have to fix atm)

Djagulingu

Germany3605 Posts

November 09 2016 15:23 GMT

#15882

On November 09 2016 23:00 Manit0u wrote:

Show nested quote +

Django being opinionated isn't such big of a problem (depending on what you want to do). Single request generating 1.6 million db queries is (something I have to fix atm)

One single http request generating 1.6 million db queries? That's some big number.

Uldridge

Belgium4951 Posts

November 09 2016 15:35 GMT

#15883

So I'm just programming here and there, kind of a novice, I like it, doing it as a sideproject for some stuff and for more serious things for some other stuff..
I like functions and I've written a script to sort and selectively take parameters out of a huuuuuuuuge csv file (500+k lines, 50mb; I mean it's huge for me at least). The program works and has been pretty optimized, even though it's just a small script (217 lines, 10+ functions). I, however, think (or even know) that encompassing this in a class would be preferable, but I can't really find any reason to do this, because my script works fine. If you think I should put this in a class (or different classes), why should I?

Also, because I don't have any more courses on programming/algorithm design/... I don't really have a great inclination to learn alot more to make myself a better programmer (I have a very slow trial and error process, but I'm getting better I guess). I'm just wondering howmuch more should I invest into getting to learn new things or better ways of coding? My current knowledge/skills end with classes. Recursive functions, lambda expressions, decorators are kind of on my to do list. I also have some more commands/modules I'd like to learn that are on the list. What are, in your opinions, obligatory things I have to get skilled at/know to work with that you consider very important to become a.. let's say decent coder.

I work in Python and think of using it for functional things, not anything graphical per se, but I'm not ruling that out.

mantequilla

Turkey779 Posts

November 09 2016 15:51 GMT

#15884

1st day wordpress question, it's about a plugin but is a fundamental question about wp architecture:

now there's a plugin called 'polylang', is to show pages-posts in different languages. it has a menu called 'lang switcher'. if you put this menu on the main page it shows flags of those languages, clicking on them leads you to the page in that language.

question is:

for the template creator, is it required to put this menu somewhere in the template?

I am writing a simple template myself so I was able to call the function 'pll_the_languages' of the plugin which generates language menu, but if I was using a premade template the creator must have called the function in the template, or I would be required to edit the template myself?

suppose there are 5 different language plugins on the marketplace, theme creators must consider all of them, for their template to work with all of them?

if a non technical user can't say put something (lang switcher) on the upper right side of the page without editing the template, templates are not much more than eye candy+layout right?

AKnopf

Germany259 Posts

November 09 2016 16:46 GMT

#15885

On November 10 2016 00:35 Uldridge wrote:
So I'm just programming here and there, kind of a novice, I like it, doing it as a sideproject for some stuff and for more serious things for some other stuff..
I like functions and I've written a script to sort and selectively take parameters out of a huuuuuuuuge csv file (500+k lines, 50mb; I mean it's huge for me at least). The program works and has been pretty optimized, even though it's just a small script (217 lines, 10+ functions). I, however, think (or even know) that encompassing this in a class would be preferable, but I can't really find any reason to do this, because my script works fine. If you think I should put this in a class (or different classes), why should I?

Also, because I don't have any more courses on programming/algorithm design/... I don't really have a great inclination to learn alot more to make myself a better programmer (I have a very slow trial and error process, but I'm getting better I guess). I'm just wondering howmuch more should I invest into getting to learn new things or better ways of coding? My current knowledge/skills end with classes. Recursive functions, lambda expressions, decorators are kind of on my to do list. I also have some more commands/modules I'd like to learn that are on the list. What are, in your opinions, obligatory things I have to get skilled at/know to work with that you consider very important to become a.. let's say decent coder.

I work in Python and think of using it for functional things, not anything graphical per se, but I'm not ruling that out.

It really depends on where you want to go with your programming. How "serious" is your serious stuff.

I'd like to highlight a sentence that made my guts itch a little "[...] but I can't really find any reason to do this, because my script works fine". A piece of software is not good because it works (though it has to work in order to be good). There are many different takes on software quality but if you are using Python you are going in the direction of application programming and in this context, software must always be evolvable. This is because the place where the software lives is most likely to be changing over time. Maybe the file format will change or the desired output. Maybe some other pieces will want to make use of your script, too, and you will need to add more parameters. Making this kind of changes should be easy to do. And not only for yourself, but for anyone who has never seen your script before. If you leave one of your serious projects, someone else will have to make these changes. In order to do so, he will need to understand the necessary parts of the 217 lines of code you wrote. Depending on your style of coding, this might be an easy task or a very difficult one. Your goal in programming should always be to make the task of understanding and altering/extending your code as easy as possible for others. You always write code for other coders, not for yourself.
Now what does that have to do with classes? Classes usually encapsulate a state and behaviour that is depending on that state. Granted your file contains data for people and you are looking for the first person with the name of "Alan". I guess you have some kind of variable that holds the state on how for into the file you have been looking so far. Maybe you read the while file at once, but then you will get problems with bigger files.
With classes, it's mucher easier to read and understand the scope of this "pointer" variable. It's only visible in functions from that same class and its scoped to the instance of the class. You do not have to worry, if some other piece of softwarethat loaded your file, has also access to this variable and if you do it right, the writer of that other piece of software does not even need to know of its existance. You made using your script easier for him.
Since your script may not have all that much to do with state, I think the behaviour aspect of classes might be more interesting for you. Classes are a way of tidying up your functions into nice, fitting name spaces. Imagine all your clothes were in one huge box in the corner of your bedroom. You would need to search the whole box to find what you need (say, a shirt). But if you have a proper wardrobe with drawers for socks and hangers for shirts. This way, it is much easier to anyone, who has never been in your bedroom to fetch a shirt for you. :-)
What does this mean for your csv readerscript?
First of all, you could wrap all your functions into a class that may have the name CsvReader. This is a first step into the right direction. When you think about how your program works you might find that exporting csv files consists of different subtasks, such as:


- fetching user input (maybe?)
- finding files
- detecting file encoding
- handling file errors
- accessing and providing the actual data
- render the data
  - maybe as text in the console
  - maybe as export in another format such as
    - html
    - xml
    - json

You will find, that each of your 10 functions falls into one (or in a bad case, multiple) of those categories. Now you found useful submodules to your csv export module.

Usually, it's pretty easy to come up with some code, that works. But to make it easy to understand and easy to extend and easy to fix (cause, doh, we all make mistakes and someone is gonna have to debug your shit :D) is were you have to put effort into it.
If you do it right, you make solving similar problems much more easier, because you can reuse code, you have written bevor. Say you want to read xml files instead. Which submodules would still work the same?


- fetching user input
- detecting file encoding
- handling file errors
- render data

So the only thing you would need to write to make your csv reader into is the one submodule that is responsible for accessing and providing the actual data. Now, isn't that cool? :-) And even better: All your exporters are available for free, too. So you can export any xml as html or json, without having to write a single line of code (if done correctly, that is).

As you might have guessed by now, this is a rather complex task and it cannot be done justice in one TL-post. I tried to explain by hand what many others have explained before, but I thought this might a bit more engaging than a link to wikipedias page about software architecture or an amazon link to Robert C. Martins "Clean Code". I guess if you are interested in writing beautiful and elegant solutions that are easy to understand and maintain for the guys to come after you, then you will find good resources on your own. A wild guess: There might even be some good posts about this in the very first post of this thread.

Cheers!

P.S.: Please reread my very first sentence. If you just like fooling around and have coding some stuff that works, than there is nothing wrong with that! I congratulate you to a fun and satisfying hobby. But this has not much to do with a more serious approach to programming.

Djagulingu

Germany3605 Posts

November 09 2016 16:58 GMT

#15886

A few reasons for turning your script into a class and expose a programming interface for the class:
1- Because you want to be able to use that script in other applications using only one import statement and a function call.
2- Because you want to be able to download that script into other places using only "pip install super-awesome-csv-parser-python-script-that-works-fine" instead of copying and pasting that file everywhere.
3- Because other people also would like to be able to download that script into their projects just by saying "pip install super-awesome-csv-parser-python-script-that-works-fine" and use its functionality with an import statement and a function call.

50 mb is nothing btw. Try a few gigabytes.

On November 10 2016 00:35 Uldridge wrote:
Also, because I don't have any more courses on programming/algorithm design/... I don't really have a great inclination to learn alot more to make myself a better programmer (I have a very slow trial and error process, but I'm getting better I guess). I'm just wondering howmuch more should I invest into getting to learn new things or better ways of coding? My current knowledge/skills end with classes. Recursive functions, lambda expressions, decorators are kind of on my to do list. I also have some more commands/modules I'd like to learn that are on the list. What are, in your opinions, obligatory things I have to get skilled at/know to work with that you consider very important to become a.. let's say decent coder.

I work in Python and think of using it for functional things, not anything graphical per se, but I'm not ruling that out.

You should really have an inclination to learn a lot more to make yourself a better programmer. That will, in return, make you a better coder because you'll organize your code better (instead of haphazardly cranking out code. 217 files and 10 functions for a CSV parsing python script? damn. I don't really know what you're doing, but that shouldn't take 217 lines imo).

mantequilla

Turkey779 Posts

November 09 2016 17:32 GMT

#15887

isn't programming pragmatic in nature? you do what works out for you, but be aware of its immediate limitations. I could say why don't you package your python code as jyhton jars so java people can use your csv parser too? I could use a good csv parser in java but there's no benefit for you doing it

little curiosity + need, otherwise jumping into some libraries just to learn them turns out unfruitful often.

RoomOfMush

1296 Posts

November 09 2016 17:41 GMT

#15888

On November 10 2016 02:32 mantequilla wrote:
isn't programming pragmatic in nature? you do what works out for you, but be aware of its immediate limitations. I could say why don't you package your python code as jyhton jars so java people can use your csv parser too? I could use a good csv parser in java but there's no benefit for you doing it

little curiosity + need, otherwise jumping into some libraries just to learn them turns out unfruitful often.

Thats how people in the 80ies thought and see what it got us. Unmanageable code and bugs that are impossible to find.

mantequilla

Turkey779 Posts

November 09 2016 18:22 GMT

#15889

but he's just a guy writing some python scripts for personal use and isn't even a professional. I'm not saying do hacky things to make it work when writing an enterprise app. No point for instant negativity, although I'm used to it among programmer fellas.

ZigguratOfUr

Iraq16955 Posts

November 09 2016 18:25 GMT

#15890

On November 10 2016 03:22 mantequilla wrote:
but he's just a guy writing some python scripts for personal use and isn't even a professional. I'm not saying do hacky things to make it work when writing an enterprise app. No point for instant negativity, although I'm used to it among programmer fellas.

That's also something people in the 80s thought. Besides hacky things have ways of back-firing even when it's only for personal use.

mantequilla

Turkey779 Posts

November 09 2016 18:32 GMT

#15891

then I should have answered like this:

grab a couple "enterprise app development in python", "scalable app architecture", "oop design patterns", "functional programming in python", "algorithm analysis" books. then refactor your shit into microservice architecture.

is this more 2000's?

Uldridge

Belgium4951 Posts

November 09 2016 19:16 GMT

#15892

Thanks for the answers. To clarify, it's parsing a file, and on every line is a parameter with a certain value corresponding to a name. This name comes in the file multiple times, but with different variables. I have to get the right variables (multiple) with the corresponding values that are unique to this name and put them in an output file. This is the main thing, there is some more stuff where I have to join other values from other files to that output file. It's pretty straightforward, but for someone who had only 1 course as an intro to programming, it wasn't easy.. I had to rack my brain to find a good solution, but ultimately I think I found a very elegant solution to it and it was very satisfying. It's a script I made for a plant's genetics thing, so I guess you could say it's in a semi professional context lol.

I am looking forward to slowly becoming better at programming though, it's very satisfying stuff!

By the way, 500k lines is nothing? Howmany lines do 2Gb files have?!

mantequilla

Turkey779 Posts

November 09 2016 19:36 GMT

#15893

if the file is ascii encoded 1 byte is 1 character so, 2gb is 2 000 000 000 bytes, divided by 500k, should be 20 000 characters per line.

if you are not loading the whole file into the memory, size shouldn't matter.

a better solution that comes to me is, import the file into an sql database so you can leverage database's services like querying, indexing, caching etc. without worrying about working with the file itself

it would make it easier to form relations between records too. plus an opportunity to learn about databases

http://stackoverflow.com/questions/2356851/database-vs-flat-files

depending on your domain cvs might be better too

Djagulingu

Germany3605 Posts

November 09 2016 20:12 GMT

#15894

On November 10 2016 04:16 Uldridge wrote:
Thanks for the answers. To clarify, it's parsing a file, and on every line is a parameter with a certain value corresponding to a name. This name comes in the file multiple times, but with different variables. I have to get the right variables (multiple) with the corresponding values that are unique to this name and put them in an output file. This is the main thing, there is some more stuff where I have to join other values from other files to that output file. It's pretty straightforward, but for someone who had only 1 course as an intro to programming, it wasn't easy.. I had to rack my brain to find a good solution, but ultimately I think I found a very elegant solution to it and it was very satisfying. It's a script I made for a plant's genetics thing, so I guess you could say it's in a semi professional context lol.

I am looking forward to slowly becoming better at programming though, it's very satisfying stuff!

By the way, 500k lines is nothing? Howmany lines do 2Gb files have?!

I don't remember the exact number but the number has 8 digits. I had an input CSV, which was 248mb and it has a point of departure, a point of arrival and time of departure of busses. Using a routing machine and gtfs data of my city, I calculated which stop the bus was at what time and outputted that as a CSV. This happened very recently too. It was an enlightening experience for me.

My point was, when you work on low number of lines (at 50mb, 500k lines is a low number. it means you have a low number of columns), you can get away with bad programming. You can do everything on memory and get away with it. What works with small file sizes might not work at all for large files. My first script worked real fine for a few thousand lines, didn't work at all for what it was intended. I had to do a whole lot of modifications.

Deleted User 3420

24492 Posts

November 09 2016 20:36 GMT

#15895

I have a question about understanding singletons

first question, is singleton merely an abstract concept?

second question, in java, is this how I use it


private static Object SINGLETON = new Object();

//empty constructor for my object
Object() {
}

//getter for the singleton
public static <*insert object state data*> getInstance() {
return SINGLETON;
}

If this is accurate, is it correct to say that the getter is just used whenever I want to access the singleton's data?
But if I wanted to assign something *as* a singleton I would still do like Object BlahBlah = SINGLETON; ?

Also, can someone explain the line


public static <*insert object state data*> getInstance()

In an example I am looking at, there are multiple parameters inside the < >. Is that just something you can do in java that I didn't know about? Why is the return type the parameters, instead of just the object type?

mantequilla

Turkey779 Posts

November 09 2016 20:49 GMT

#15896

first question, is singleton merely an abstract concept?

I'd say its a highly technical one. Often you need to have only 1 instance of an object, reason may be related to multithreading, or it's an expensive object to create and no more than just 1 is required. Although nowadays its a better practice to use a DI injector like guice or spring framework to create such instances, since singletons have problems with testability and maintainability.

extra info: https://en.wikipedia.org/wiki/Singleton_pattern#Lazy_initialization

its called generics. It allows some class/interface to work with an arbitrary type instead of a known class or interface. Such as List<T>, it allows you to have list of whatever you want. Otherwise if it was List<SomeInterface> everything you need to be put in a list would require implementing an interface (plus you won't be able to differentiate between lists of different types)

Blitzkrieg0

United States13132 Posts

November 09 2016 20:54 GMT

#15897

To add onto that, your constructor needs to be private. Right now you're at default access.

ZigguratOfUr

Iraq16955 Posts

November 09 2016 20:56 GMT

#15898

On November 10 2016 05:36 travis wrote:
I have a question about understanding singletons

first question, is singleton merely an abstract concept?

second question, in java, is this how I use it


private static Object SINGLETON = new Object();

//empty constructor for my object
Object() {
}

//getter for the singleton
public static <*insert object state data*> getInstance() {
return SINGLETON;
}


public static <*insert object state data*> getInstance()

A singleton is just a design pattern (one which I personally think is horrible and overused but let's not get into that). As for your code the constructor(s) should be private since the whole point of a singleton is to only have a single instance. As for the <*insert object state data*> I don't know, it's usually just the singleton class. What's your example?