> I honestly wouldn't want to have to write this stuff in Python for a simple reason: I don't think I could live without static typing, which is a fantastic tool when you need to manage a large code base written by multiple people over multiple years. I can make a change in some package, do a dry-run compile of every system that uses it, and then see what needs updating. It gives me certain guarantees about data integrity right at compile time, which is super helpful when you're doing data conversion.
Programming in the large without type safety is a fool’s errand.
> But hey, different jobs, different tools.
Exactly. There’s a reason your kitchen drawer isn’t full of just sporks.
> Programming in the large without type safety is a fool’s errand.
Lol. Right. No big system has ever been built in an untyped or weakly typed language. Well, except just about every bit of software we all use everyday. But it does seem like some small startups can't get by without it.
>No big system has ever been built in an untyped or weakly typed language. Well, except just about every bit of software we all use everyday. But it does seem like some small startups can't get by without it.
Many have built models of the Eiffel tower with toothpicks too, so?
You can still built things with inadequate tools: inadequate != prohibitive. You just have more problems going forward.
Which is exactly the lesson people who write large scale software have found.
What is this "just about every bit of software we all use everyday" that you wrote about as been written in weak types?
Most major software is still written in C/C++ (anything from operating systems, Photoshop, DAWs, NLEs, UNIX userland, MS and Open Office, databases, webservers, AAA games, what have you). One could use just that C/C++ software, and they'd have almost all bases covered.
The rest is e.g. Electron based software and online services. For the latter, most of the major ones (e.g. Gmail, Apple's iCloud services, Microsofts, online banks, online reservations, etc, etc) are not written in "weakly typed languages", only the client is.
And those that were initially written in a weakly typed language, e.g. Twitter with Ruby on Rails, others with Python, etc, have rewritten critical services (or entirely) to statically typed languages (e.g. Twitter went for Java/Scale, others for Go, etc).
And even for the client, most shops are now turning to Typescript (and FB to Flow) because they've found weakly typing is not good enough for large scale. So?
Python is not weakly typed. It is strongly typed in that it forbids operations that are not well-defined. For example, adding a number to a string) rather than silently attempting to make sense of them. I agree wholeheartedly about weakly typed languages, though.
I believe that marketing Python as "strongly typed" has the potential to confuse rather than educate. Python still crashes at runtime with these errors. It has nice error messages, but it still crashes, potentially in production. If you want to create your own "types", you'll have to add your own runtime checks. It's much more sane than JavaScript, but it's not strongly typed like Haskell. Python does not automatically coerce some built-in runtime values, that's it.
Not automatically coercing values is all that strong typing means. Getting a type error before you run the program is static typing. They're separate axes, and both useful to talk about in a language.
Could you elaborate or point to a resource? AFAIK, term "strongly typed" is usually used to refer to that the type cannot change but I'm failing to find a well defined definition or the comparison against statically typed.
Static typing means that types are figured out statically by looking at the source code, and type errors are detected then when it notices a mismatch. Dynamic typing means that types are worked out at runtime by looking at live objects when code operating on them executes.
Strong typing means that types cannot be substituted for other types. In C, you can write `int x = "one"` and the char * (address of) "one" is automatically converted to an int, or in Javascript you can write 1 + "2" and a string "1" is automatically created; depending who you're talking to, either or both of these qualify as weak typing.
They're both spectrums, and commonly confused with each other.
You're explaining static typing vs dynamic typing. I'm still failing to see how different Strong vs Static. If the only difference is "Static" means "types are figured out statically by looking at the source code" do you mean it's possible to change the type unlike strong typing? If not, can we say Static encapsulates Strong?
Static typing is not a superset of strong typing, they're on different axes. Strong vs weak typing (which I explained in the second paragraph) is about how strictly types need to match expected types before you get a type error. Static vs dynamic typing is about when you get a type error (during a static typechecking phase, or at runtime when you try to use a value as that type).
When you say the type cannot change, that's ambiguous: do you mean the type of the value a variable holds, or the type of the value itself? In C (a statically typed language), "int x" means that x will always hold an int, but you can still assign a pointer to it, it just turns into an int (weak typing). In Python (a dynamically typed language), the variable "x" wouldn't have a type (so it could hold an int at one point and a string later), but the value it holds does, and because it's strongly typed, it would throw a type error if you attempted to use it in a place where it wanted a different type (eg, `1 + "2"` does not turn 1 into a string or "2" into an int).
If I got this correct, you're saying strong can be compared to weak and static can be compared to dynamic. So there is no such thing as strong vs static typing comparison.
"Dynamic typing" is really just case analysis at runtime. Every static language is capable of dynamic typing, it's not some feature that statically typed languages lack. A dynamic language is really just a static language with one type.
Because most statically typed languages allow us to define our own types, add type signatures to constrain etc. Dependently typed languages also allow types to depend on values. Inference is useful, but only one aspect of static typing.
My point is that your marketing is misleading. Use "strong dynamic types" if you must, but for Python, it would be more accurate to say "strongly tagged".
C's typing is so week it might as well be an untyped language - not even a dynamically typed langue. And that's what most of the software you run every day runs on.
Static typing was all the rage 20 years ago. C++ and Java were going to save us from the chaos of C. What people found was the vast bulk of software defects are not problems that can be detected by static typing.
Static typing just created a constraining, inflexible code base, that was no more reliable than C or smalltalk or lisp. Once your beautifully conceived collection of types were demolished by the cold hard reality of changing business requirements the type system actively worked against you.
Python and ruby and javascript started gaining traction, and at first it seemed crazy to use a language that didn't have a static type checker. But after people started using them they realized they just didn't have the kinds of bugs that a static type checker would catch anyway - because those types of bugs are caught by the dynamic type checker (something C doesn't have, and C++ only sort of kind of has) at run time when you write tests. And writing tests also caught all kinds of other logic bugs that didn't have anything to do with types. They were writing software faster and more reliably in dynamically typed langues than they ever could in the old statically typed languages.
Of course no language is a silver bullet, and writing software is still hard. Combine that with the fact that our industry has no sense of history, and a fair number of programmers today have only used dynamically typed languages, and you can see why the static typing fad is coming back around.
It seems intuitive that caching these type errors at compile time rather than run time will make for a more reliable system. But history tells us otherwise. Unless you just don't run your code before pushing it to production the dynamic type checker will catch it just as well when you run tests. And your types will drift away from the reality of the business requirements grinding development to a halt.
The static typing fad has a 5 year shelf life. Just enough time for managers to force a new generation of programmers to re-write all their code in typescript or whatever and learn it is just as unreliable, and much harder to work with.
(Sound) Type systems guarantee correctness for the invariants encoded as types. If it compiles, you know it doesn't have any type related errors at all. With more evolved type systems even your program's logic (or large parts of it) is guaranteed.
Tests just allow you to test random invariants about your program. If it compiles and your add() method works when passed 2, 2 and gives 4, it still might not work for 5, 5... (contrived example: imagine it with much more complex functions, though even a simple e.g. "one line" time conversion can have similar issues).
You need to test anyway. So, is it the case that type systems provide much value beyond what a proper set of tests, which are necessary, are going to provide anyway?
If you skimp on testing your system will be crap, but at least the type system can fool you into thinking otherwise because it still compiles.
Actually, if your type system is powerful enough, you don't need to test. That's the source of the "if it compiles, 99% of the time it works right" people mention about Haskell (and even more so languages like Idris etc).
Type systems are tests -- just formal and compiler-enforced, not ad-hoc "whatever I felt like testing" tests, like unit tests are.
From there on it's up to the power of the type system. But even a simple type system like Java's makes whole classes of tests irrelevant and automatically checked.
A programmer can also leverage a simpler type system to enforce invariants in hand crafted types -- e.g. your "executeSQL" function could be made to only accept a "SafeString" type, not a "string" type, and the SafeString type could be made to only be constructed by a method that properly escapes SQL strings and params. Or the same way an Optional type ensures no null dereferences.
> Actually, if your type system is powerful enough, you don't need to test. That's the source of the "if it compiles, 99% of the time it works right" people mention about Haskell (and even more so languages like Idris etc).
Types only eliminate certain tests. You will always have system tests, acceptance tests and unit tests.
One should use types to augment their system reliability.
Haskell's type system most definitely does catch some of your logical errors. That's exactly why it is so revered.
An effective use of a type system such as Haskell's Hindley-Milner can result in a vastly smaller surface area for possible problems and thus can cut a big number of otherwise mandatory unit tests off your todo list.
>Types only eliminate certain tests. You will always have system tests, acceptance tests and unit tests.
Yes, so let's eliminate them with types, instead of doing them. "Acceptance tests" are not concerned with programming.
>Types will not catch logical errors in your code.
Actually, depending on the type system, it will.
That's how program logic is tested as "proof" and programs, implementations of algorithms are determined to be logically correct in more exotic languages (but even in C + some restrictions + the right statically checking tooling, NASA/JPL style project do that).
The question is not whether a type system will catch bugs. The question is whether a type system finds enough bugs that tests (sufficient to cover the things that the type system does not catch) would not also catch.
If you have to point to something like Idris I don't think you're making a real world argument yet.
Both static type systems and unit testing are just tools which are supposed to help programmers to deliver higher quality software.
Both static type systems and unit testing have their disadvantages. For static type systems, you sometimes need to bend backward to make it accept your code and it's not very useful before the code grows large enough. For unit tests, even if you have 100% test coverage, it doesn't mean that you're safe - underlying libraries may behave in unexpected ways and the test data input won't ever cover the whole range of values that the code expects to work. Integration tests have the same problem, the prepared input represents just a few cases, plus they are generally harder to run so they are run less frequently.
So, both tools are useful but they aren't solutions for all the problems in programming. Static type systems have the advantage of being checked without running any code, which should be much quicker than running the tests. Static type systems become more useful as you increase the precision of types and the amount of annotated code in the project. When used correctly, they provide certain guarantees about the code which you can rely on and they are used to restrict the domain (set of possible inputs) of type-checked procedures and classes. This means that you can write fewer unit tests because you don't have to worry about certain conditions which the type system guards against (static guarantee of something never being `null` is quite nice).
Anyway, I think that both static type systems and tests are great tools and they can and should be used together if you value the quality of the code you write. This is getting easier thanks to gradual type systems (optional type annotations like in Python or JS) which allow you to get some of the static guarantees without insisting on everything around being typed. With tests and mypy (in Python) you're much better off in terms of code quality than if you used just one of them. I see no reason not to use them both.
> For static type systems, you sometimes need to bend backward
> to make it accept your code and it's not very useful before
> the code grows large enough.
How large need a program to become, before the advantage of being allowed to write fishy code is counter-balanced by the types becoming untractable and the code impossible to refactor in any meaningful way?
This is a serious question. Some years ago, apparently Guido Van Rossum though 200 lines would be already quite an achievement [0].
Based on my own experience, I feel that 99 out of 100 errors thrown at me at compile time are valid and would have caused a crash at runtime (ie. when I do not expect it and have lost all the context of the code change). And I get about 50 such compilation errors in a day of work, so I guess I could write without the compiler safety net for about 10 minutes. That's my limit.
One could object that a 10 minutes program written in python can accomplish much more than a 10 minutes program written in Java. That's much certain! But then we are no longer comparing the merits of compile time vs runtime type checking, but two completely different languages. Of course it is easier to write a powerful/abstract language with runtime type checks, while writing a compiler for a powerful language is much harder. Still, since (and even before) python/perl/php were invented many powerful compiled languages have appeared thanks to PL research, that are almost as expressive as script languages. So it would be unfair to equate runtime type checking with lack of expressive power.
Now of course tests are important too. Compile time type checking does not contradict testing, like you made it sound somewhat in your message. Actually, if anything, it helps to test (because of test case generators based on type knowledge to exercice corner cases).
I'm sorry if all this sounds condescending. I am yet to decide whether I should allow myself to sound condescending as the only benefit of age :)
But I'd not want to sound like I'm upset against anyone. Actually, I'm happy people have been using script languages since the 90s, for the same reason I have been happy that many smart people used Windows: my taste for independence gave me by chance a head start that I'm afraid would have been much tougher to get based on my intelligence alone.
And now that static type checking is fashionable again I'm both relieved and worried.
> Some years ago, apparently Guido Van Rossum though 200 lines
I think it's better to measure the number of separate code entities (classes and functions and modules in Python) and how many different use-cases (ways of calling functions and object constructors) each entity is expected to cover... After converting to LOC, I'd say ~500 would be the limit. After that, it's a constant fight with TypeErrors, NameErrors, and AttributeErrors - it's just that everyone is already used to this, while not many know of any alternatives. Also, there are substantial differences between languages - in some 10 lines are enough to start complaining, while in some others I've seen and worked with ~2k loc code and it was manageable.
> many powerful compiled languages have appeared thanks to PL research, that are almost as expressive as script languages.
Yes, but on the other hand, some powerful static type systems for dynamic languages also appeared, and some of them are close to Haskell in terms of expressivity. The particular example here would be Typed Racket, which has a state of the art type system which is built on top of untyped Racket. It supports incrementally moving your untyped code to the typed one (whether a module is statically or dynamically typed is decided when module is created; as you can define many (sub)modules in a single file, you can just create a typed submodule, re-export everything that's inside, and move your code there one procedure at a time). Also, it automatically adds contracts based on static types, so that they still provide some guarantees when a typed function is imported and used in untyped code. There are many interesting papers on this, and TypedRacked is really worth looking into, if you have nothing against Lisps.
> Compile time type checking does not contradict testing, like you made it sound somewhat in your message.
Damn! I actually wanted to argue exactly this: that both tools are useful and both can be used together to cover their respective weaknesses. :) Looks like I need to work harder on my writing skills...
> I'm sorry if all this sounds condescending. I am yet to decide whether I should allow myself to sound condescending as the only benefit of age :)
Well, it didn't sound condescending to me, so no prob :) But, if you'd like an advice on this: please don't try to be condescending on the basis of age alone! It's totally ok to sound condescending if you have knowledge, experience and skill to back it up... Well, at least in my book :)
> Programming in the large without tests is a fool's errand. Type systems don't guarantee correctness.
I never said you do not need tests nor that static typing is a panacea. In my view it's a necessary, but not sufficient condition, when programming in the large.
No but they help. You can find figures of a 15%-38% reduction in bugs for TypeScript versus JavaScript. So that does not consider the additional effect of strong versus weak typing.
Programming in the large without type safety is a fool’s errand.
> But hey, different jobs, different tools.
Exactly. There’s a reason your kitchen drawer isn’t full of just sporks.