Roy Tang

Programmer, engineer, scientist, critic, gamer, dreamer, and kid-at-heart.

Blog Notes Photos Links Archives About

All entries tagged software-development.

You can subscribe to an RSS feed of this list.

Jan 2017

  • Because of the nature of the web and the fact that you should never trust user input, all the validation in a web application should be done on the server side. You can additionally provide validation on the client side (via JavaScript), but this is only a concession towards a better user experience and should not be used as a substitute for server-side validation. One would think that anyone with a basic understanding of how HTTP works would understand the above easily and any failure to practice it should be considered amateur hour.

    read more (420 words)

    Posted by under post at #Software Development
    Also on: tumblr twitter / 2 / 420 words

Dec 2016

  • Cleaning up your Code

    In one of my most recent projects, a large system that had gone through a relatively long and unstable period of many, many changes due to sales demonstrations, different clients and whatnot, one of the “fun buffer tasks” I always kept around for devs was code cleanup. Because of the unstable nature of the project, there was always a lot of duplication, unused/unnecessary/obsolete classes/functions/files and so on. Unnecessarily large CSS files where most of the selectors were no longer really needed or JS libraries that weren’t actually used.

    read more (290 words)

    Posted by under post at #Software Development
    Also on: tumblr twitter / 2 / 290 words
  • Related: Learning new skills While many people working as programmers/software developers are happy enough specializing in a single programming language or platform, I generally consider it a better idea to have a wider toolset and the ability to easily pick up new programming languages as needed. The benefits should be obvious: when you have a wide variety of tools under your belt and are able to quickly learn to use a new tool, the number of work options you have increases greatly.

    read more (581 words)

    Posted by under post at #Software Development
    Also on: tumblr twitter / 0 / 581 words
  • In any reasonably large software project, the system will be so large that no one developer will have a good grasp of the details of every function in the codebase. The tendency is for developers to specialize – that is, developers tend to focus only on certain parts of the codebase and become more familiar with that part, while not having much knowledge about the other parts. This tendency is self-reinforcing – once it becomes known that the developer is an “expert” in the given module, there is a tendency that he will be assigned the most difficult and urgent tasks or fixes related to that module, further cementing his expertise.

    read more (321 words)

    Posted by under post at #Software Development
    Also on: tumblr twitter / 0 / 321 words
  • I was in a meeting once with my boss (literally the CEO, a Malaysian) and some representatives of another company (Americans) where we were discussing the technical details of a possible future partnership. At one point, one of the Americans said to my boss that he was pleasantly surprised that I was openly speaking up independently of my boss and willing to correct him on some points when he didn’t quite get the technical details right.

    read more (354 words)

    Posted by under post at #Software Development
    Also on: tumblr twitter / 3 / 354 words
  • I tried to insert the following test document:

    db.documents.write(
        {
            uri: "/test/doc1.json",
            contentType: "application/json",
            collections: "test",
            content: {
                name : "Peter",
                hobby: "Sleeping",
                other: "Some other info",
                "triple": {
                    "subject": {   
                        "datatype": "http://example.com/name/",  
                        "value": "Peter"   
                    },   
                    "predicate": {     
                        "datatype": "http://example.com/relation/",  
                        "value": "livesin"   
                    },   
                    "object": {     
                        "datatype": "http://example.com/location/",  
                        "value": "Paris"   
                    }
              	}
            }
        }
      ).
      result(function(response){
        console.log("Done loading");
      }); 
    

    Then I queried as follows:

    var query = [
      'SELECT ?s ?p ?o' ,
      'WHERE { ?s ?p ?o }' ,
    ];
    db.graphs.sparql('application/sparql-results+json', query.join('\n')
    ).result(function (result) {
      console.log(JSON.stringify(result, null, 2));
    }, function(error) {
      console.log(JSON.stringify(error, null, 2));
    });
    

    The results showed me the values of the triple, but what if I also want to get the entire document where the triple was embedded? Is it also possible to filter by other fields in the document?

  • I inserted a test document using the Node.js API:

      db.documents.write(
        {
            uri: "/test/doc1.json",
            contentType: "application/json",
            collections: "test",
            content: {
                name : "Peter",
                "triple": {
                    "subject": {   
                        "datatype": "http://example.com/name/",  
                        "value": "Peter"   
                    },   
                    "predicate": {     
                        "datatype": "http://example.com/relation/",  
                        "value": "livesin"   
                    },   
                    "object": {     
                        "datatype": "http://example.com/location/",  
                        "value": "Paris"   
                    }
              	}
            }
        }
      ).
      result(function(response){
        console.log("Done loading");
      }); 
    

    Then I tried querying it via SPARQL on the Marklogic console webapp:

    SELECT ?s ?p ?o
      WHERE { ?s ?p ?o }
    

    The query above works fine, it retrieves the test triple I inserted above

    However, if I want to filter by the literal value “Paris”, I tried the following:

    PREFIX loc: <http://example.com/location/>
    
    SELECT ?s ?p 
      WHERE { ?s ?p loc:Paris }
    

    But in this case I got zero results. Is this syntax incorrect? I was just following how the queries looked in https://docs.marklogic.com/guide/semantics/semantic-searches

    What is the proper way to do it?

  • In JavaScript, referencing variables that are declared outside of a function’s scope can be tricky. If you have code like this: var btn = document.getElementById("BTN"); var test = 1; btn.onclick = function() { alert(test); } test = 2; The click handler above retains a reference to the test variable even though it falls out of scope as soon as the script block finishes execution. When you actually click the button, the alert will show the last value of the variable when the block finished execution (2) instead of the value at the time the function was initialized (1).

    read more (361 words)

Nov 2016

  • This is a story of something I consider to be one of my worst mistakes in software product development. Some years ago I was asked whether it was feasible to write software that would be integrated with Software X that allowed us to export that software’s output into a format that was compatible with Standard Y. I took a look and after a while came back with “Well sure. We could use Programming Language M that has an API that lets us integrate into Software X so we can export the output data.

    read more (368 words)

    Posted by under post at #Software Development
    Also on: tumblr twitter / 0 / 368 words
  • I want to retrieve all documents in my MarkLogic db that have month=November and also group them by name, and get the count of records per name. I know I can get the frequency per name by using valuesBuilder with a range index on the name field, but how can I filter this result so that I only get the count of records for the month of November?

    Supposedly valuesBuilder.fromIndexes().where() can do the filtering, but I don’t know what to pass here and examples online seem to be sparse.

  • Unclear error messages

    “Button for non-service floor does not light up." For more than a decade I regularly went to an office building where the elevators verbally spouted this nonsense message whenever you tried to go to a floor that the current elevator car did not service. For context, the elevators in the building were zoned programmatically – this means that they only service a particular subset of the floors that are provided on the elevator panel itself.

    read more (579 words)

    Posted by under post at #Software Development
    Also on: tumblr twitter / 0 / 579 words
  • There was this project we had where there was a strange bug. The developer working on it found that the problem only appears when the record ID was 12. When it was 11 or less, everything was fine. When it was 13 or more, everything was also fine. After some investigation, it was found that there was some code that executed with a condition of “if record id == 12”, which was already a WTF.

    read more (278 words)

    Posted by under post at #Software Development
    Also on: tumblr twitter / 0 / 278 words
  • The software development process is already difficult mainly because a lot of it so imprecise. Requirements are often only vague wishes that the client has, with no regard to the sheer number of instructions needed to implement those requirements. Throughout the entire process it’s important to use feedback loops to determine whether development is on the right path. And like all feedback loops, their effectiveness often hinges on how quickly we are able to turn around and give and incorporate feedback into future iterations

    read more (592 words)

    Posted by under post at #Software Development
    Also on: tumblr twitter / 0 / 592 words

Oct 2016

  • It Should Be Easy, Right?

    In Tagalog: “Madali lang naman diba?" Probably one of the most annoying things a programmer can hear, especially from a client or a manager who has no appreciation of how complex software development is. It’s presumptuous at best and actively damaging to schedule and morale at worst. We already know estimation is hard, there is no need to make it more complicated by automatically assuming the best-case scenario (or in many cases, an impossible scenario)

    read more (356 words)

    Posted by under post at #Software Development
    Also on: tumblr twitter / 1 / 356 words
  • “Composition over inheritance” is an object-oriented programming principle that I’m sad to say many devs I’ve encountered aren’t too familiar with. Composition provides greater flexibility, modularity, and extensibility in large software systems as compared to inheritance, especially for statically typed languages like Java that don’t support multiple inheritance The most common examples of the problems caused by too much inheritance involved generic object such as the game objects example in the wikipedia page linked above.

    read more (675 words)

    Posted by under post at #Software Development
    Also on: tumblr twitter / 1 / 675 words
  • Coding Frameworks

    For the better part of my software development career so far, I’ve had the doubtful pleasure of being one of the devs using and maintaining our in-house web development framework. Framework coding is a bit different from the actual application development. At the core it’s a simple idea: you have a whole bunch of code that helps do programming tasks that you expect will often be necessary in a certain set of projects, so you write that code with the intent of reusing it across multiple projects.

    read more (844 words)

    Posted by under post at #Software Development
    Also on: tumblr twitter / 0 / 844 words
  • A Programmer's Hubris

    “We will encourage you to develop the three great virtues of a programmer: laziness, impatience, and hubris.” — Larry Wall, Programming Perl (1st edition) Hubris is a fancier word for an excessive sense of pride. Why is this to be considered a great virtue for a programmer? Programming is at least partly an act of creation, which means there is an element of craftsmanship involved. A craftsman imbued with hubris is able to take pride in his work – he is driven to create work that is the best quality he can provide.

    read more (403 words)

    Posted by under post at #Software Development
    Also on: tumblr twitter / 0 / 403 words

Sep 2016

  • Type Interfaces Matter

    … especially for strongly-typed languages. In one of the bigger Java projects that I took over, I was often annoyed to find some devs had written method signatures like public void doTheThing(HashMap<K, V> params) Which is silly – not because of the naming, that’s obviously not a real-world function name. The silly part is requiring a particular implementation (HashMap) instead of the generic interface (Map). It unnecessarily restricts your API and makes it less flexible.

    read more (164 words)

    Posted by under post at #Software Development
    Also on: tumblr twitter / 0 / 164 words
  • Overhead scale up rapidly as project team size increases. Every time you add a new person to the team, he comes with a lot of overhead such as the need to learn the project details, responsibilities of other team members, who to consult when there’s trouble, custom project procedures, and so on It’s a reinforcing cycle too. As overhead increases, the team imposes more processes and restrictions to make sure everyone is doing the right thing and there are no screw-ups.

    read more (314 words)

    Posted by under post at #Software Development
    Also on: tumblr twitter / 0 / 314 words
  • Overtime

    Overtime in software development projects seems to be a given. Sure, there are projects and companies that don’t need it, but those feel like the exception rather than the norm Overtime in software development is a natural consequence of schedule pressure and the fact that estimation is hard, which is why it’s understandably common, but that doesn’t mean we shouldn’t try to avoid it More than once I’ve been in a situation where the team stays overnight to try to get a build or release ready for the next day only to run out of time and have to delay the deployment anyway.

    read more (368 words)

    Posted by under post at #Software Development
    Also on: tumblr twitter / 0 / 368 words
  • Will AI replace all our jobs?

    A while back, news came out that an AI was able to perform a diagnosis after the human doctors were stumped. Having just finished binge-watching the entire run of Star Trek Voyager, I immediately thought that was the sort of AI advancement that could lead to such things as Voyager’s Emergency Medical Hologram, an AI which could replace a human doctor. Of course, it is still very unlikely for this to happen anytime soon, medicine is a very complex field and new things are still being discovered all the time.

    read more (491 words)

    Posted by under post at #Software Development
    Also on: tumblr twitter / 0 / 491 words
  • Without a doubt, the number one problem in software development projects is **schedule pressure, **that is, the pressure to meet unreasonable deadlines and targets Almost all other problems can be overcome if there were no schedule pressure: weaker developers could be mentored to become better, more productive, and commit less faults less faults overall will be committed anyway if there were no schedule pressure poor requirements could be threshed out in more detail difficult clients can be argued, worn-down, and eventually reasoned with problematic team members can be counseled, or replaced with new blood and so on But of course, the reality is that we live in a world with deadlines and targets, many of them set by people who have no idea about the complexities of software development.

    read more (396 words)

    Posted by under post at #Software Development
    Also on: tumblr twitter / 0 / 396 words

Aug 2016

  • Estimation is hard. Estimation has long been the bane of many software developers and software development projects. But there are two secret ways to be able to produce perfect estimates for software development work all the time! One is dependent on talent, and the other is dependent on technology Psychic precognition, i.e. be able to predict the future Have a time machine, so you can go back in time and tell yourself how long the work would have taken Such precognition is necessary to have perfect estimates because of all the unknowns present at the start of a software project.

    read more (794 words)

    Posted by under post at #Software Development
    Also on: tumblr twitter / 0 / 794 words
  • I like to say that software development is a challenging career because no two projects are ever the same and there are always new challenges to face and new concepts to learn, but the truth of the matter is a bit more complex. Writing software is about breaking down large problems into a series of very small technical problems for which we already have solutions. Examples of small enough technical problems include list sorting, comparison, arithmetic operations, path traversal, string concatenation, returning a string as an HTTP response, rendering text to the screen, retrieving submitted parameters from an HTTP request, and so on.

    read more (730 words)

    Posted by under post at #Software Development
    Also on: tumblr twitter / 0 / 730 words
  • I was originally going to write a post about the problems development teams face as they get larger, but the section on development standards was long enough by itself so here we are. Having some sort of development standard in a project development team becomes a lot more important as project size goes up (for obvious reasons). There are different kinds of standards to consider, but generally I break them down into design standards and coding standards.

    read more (1174 words)

    Posted by under post at #Software Development
    Also on: tumblr twitter / 0 / 1174 words
  • Pros: It is a very rewarding career financially. Software development often ranks in the top 10 highest-earning careers in most countries There is a lot of scope – you could be developing web applications, mobile applications, embedded applications, client-side, server-side, data analysis, artificial intelligence, games, etc It is very difficult to be bored. You can always automate away the boring stuff. Different projects always present different challenges. The field is evolving rapidly so there are always new things to learn.

    read more (397 words)

    Posted by under post at #Software Development
    Also on: twitter / 0 / 397 words

Jul 2016

  • Evaluation of programmer performance is notoriously hard. You want to be accurate and at the same time fair such that all programmers on your staff are evaluated in the same matter. However, there are no good, objective, universally accepted standard metrics. It follows from the fact that there are no good, objective, universally accepted standard metrics for program size. Typically each programmer in a team will not be doing the same task or even the same type of task, so in order to produce fair evaluations you will need some standard metric of program size to normalize any evaluation.

    read more (642 words)

    Posted by under post at #Software Development
    Also on: twitter / 0 / 642 words
  • Learning to SQL

    ****Since I wasn’t a graduate of computer science, there were many concepts of software development I really only got exposed to when I started working. One of those was the concept of a relational database, and hence SQL. The company I worked at gave all new hires a training regimen that started with about a week of SQL. Despite not knowing anything about it beforehand, I took to it like a mouse takes to cheese.

    read more (1137 words)

    Posted by under post at #Software Development
    Also on: twitter / 0 / 1137 words
  • Certifications

    I never really put much stock in certifications. I felt that they were no guarantee of knowledge or expertise, and that many people who did have knowledge or expertise wouldn’t necessarily have a certification to say so. Add to that it often seem overpriced to even apply for the certifications, so I didn’t have a high opinion of them. That being said, I have had the opportunity to take professional certification exams twice in my life (both luckily paid for by my employer at that time).

    read more (945 words)

    Posted by under post at #Software Development
    Also on: twitter / 0 / 945 words
  • A friend of mine had an informal consultation with me the other day (read: asked me questions over FB messenger) about what their IT staff was telling them about a file upload vulnerability that had been recently exploited in one of their applications. Obviously it was difficult for me to judge given that I didn’t know all the details, but for me it was most likely a vulnerability introduced in the application code itself.

    read more (649 words)

    Posted by under post at #Software Development
    Also on: twitter / 0 / 649 words

Jun 2016

  • Some time ago, one of my many intrepid followers pointed out that this blog tested poorly on web page performance according to this Speed Testing Tool. Now, I’m of the opinion that for a personal blog such as this, web performance isn’t really a mission-critical sort of thing, but as a software developer who has often had to work hard to optimize the web applications we delivered to our clients, it kind of became a matter of pride :p

    read more (924 words)

    Posted by under post at #wordpress #Software Development
    Also on: twitter / 0 / 924 words
  • Working with Testers

    I had my first taste with working with software testers during my first project where I was involved with porting an old system to a new version of the software. My first task involved porting reports which were to be generated by the users then printed out. The task wasn’t too difficult: basically you took the source code of the report (it was some weird binary format recognized only by the particular reporting tool – that was how it was done back in the day) and open it using the newer version of the tool, and the tool did some sort of migration magic to adapt it to the new format, then you just save it back again.

    read more (1148 words)

    Posted by under post at #Software Development
    Also on: twitter / 0 / 1148 words
  • In the modern era of online services and applications, it is getting more and more common to hear of databases and systems being hacked and user data being exposed. The most dangerous of this data is the user’s password since it may allow access not only to your own service but to other services as well. As an application developer, the below is probably the bare minimum you need to know when handling user passwords:

    read more (695 words)

    Posted by under post at #Software Development #Tech Life
    Also on: twitter / 1 / 695 words
  • More stories from the early days. Evaluating someone’s programming ability is hard, especially someone fresh out of college. A student’s grades is in no way indicative of how well he can program after all. So most nontrivial programming jobs have some sort of complicated application process involved. I remember going in and taking an exam. Most application processes will have some sort of written exam to filter out people who look good on paper, but can’t actually do anything.

    read more (913 words)

    Posted by under post at #Software Development
    Also on: twitter / 0 / 913 words
  • You Are Not Your Code

    I once had to advise someone who found himself irritated at receiving lots of comments during code review. I think my response was good enough to quote verbatim: Remember: You are not your code. You are not the hundred or so lines of C or Java or JavaScript or whatever that you wrote today. This problem arises because you are too attached to your code. Your ego is associated with the code you write and you feel that any comments or defects found reflect upon you as a person.

    read more (329 words)

    Posted by under post at #Software Development
    Also on: twitter / 3 / 329 words

May 2016

  • Origin Story

    A while back I answered a question over on Quora about how I got started down the path of programming. It’s not a particularly interesting story, but I still thought I’d record it here for posterity. Sometime when I was much younger, maybe somewhere between twelve to fourteen years old, I remember having some sort of QBasic programming learning book at home. I forget how we got it, I think my uncle brought it home for me sometime for some reason.

    read more (1031 words)

    Posted by under post at #Software Development
    Also on: twitter / 0 / 1031 words

Oct 2015

  • Things software development projects can learn from “The Martian” (maybe spoilers lol):

    • If you rush things, you might leave something behind that will cost you a lot more later
    • You need to have buffer in your schedule for the inevitable disasters
    • If you have impossible deadlines, limited resources, overworked staff, and decide to skip testing to meet deadlines, don’t expect things to go smoothly
    • If your stuff doesn’t work, see if there’s some Chinese alternative. Or look for some sort of coffee-powered genius to come up with a miracle solution
    • If you make a unpopular decision, your staff might go off on their own and risk their lives trying to complete the mission
    • Problems will occur at almost every step, be prepared to make stuff up as you go along
    • Solve one problem at a time until you get to go home

Sep 2015

  • I have installed an SSL certificate on my Apache server, but when I access the site via URL from a different machine, an HTTPS error is shown and viewing the certificate details says “this certificate has an invalid digital signature”

    If I view the same URL from within the server itself, the certificate is fine and there is no HTTPS error.

    I’m not sure what to look for in httpd.conf. Any advice?

    Thanks!

Aug 2015

Jun 2014

May 2014

Feb 2014

Nov 2013

  • I want to add focus and blur handlers to all CKEditor instances in our web application. I would like to add the handlers in one place, instead of hunting down every part where we instantiate a CKEditor. Can this be done, like maybe in the config.js editorConfig setup?

    I can’t do something like “on document ready, add handlers to all CKEditor instances on the page” either, since additional editor instances may be created dynamically.

Oct 2013

Sep 2013