GET /wp-includes/js/dist/vendor/lodash.min.js?ver=4.17.19 net::ERR_ABORTED 403 (Forbidden)

Looking at the site errors logged by Apache, it shows:

[Thu Jan 13 01:50:48.844763 2022] [access_compat:error] [pid 18887] [client 50.47.132.16:60932] AH01797: client denied by server configuration: /var/www/html/aresou.net/public_html/wp-includes/js/dist/vendor/lodash.min.js, referer: /wp-admin/index.php

Weird enough, this error only happens on admin pages, not on the front pages or posts. The first thing I did is to check the folder and file permissions within the WordPress installation. Apache might be unable to serve the static file because www-data account cannot read that file or folder.

Nothing suspicious after checking that. I even set folders to 755 and files to 644 with chmod. Seems like the problem isn’t in the file system. Having done a bit more research, I realized that the next possible cause is the Apache site configurations.

I have three sites on my host, serving two blogs (one of them is aresou.net, aka this site you’re reading) and one personal wiki. The wiki shares the same root domain with this blog. First I reviewed the <Directory> sections of the configs for the two sites with their own domains. Everything looks good, there was no permission restricted from either root or sub folders. Apache should not be blocking access to these folders.

As the wiki site is sharing the same root domain with this blog, I did not think of checking the config. Well, this is the worst assumption I had in the debugging. With more research, I found a line somewhere saying “Apache will combine directives from all enabled site configurations when loading”. To be precise, let me quote from Apache official documentation:

Directives in the configuration files may apply to the entire server, or they may be restricted to apply only to particular directories, files, hosts, or URLs. This document describes how to use configuration section containers or .htaccess files to change the scope of other configuration directives.

https://httpd.apache.org/docs/2.4/sections.html

Which means, any directives from any site config file will be applied to the whole server, if it is not restricted to a particular scope. I immediately went back and checked the config for my wiki site. There were a few lines blocking access for some folders, but not everywhere:

...
Require all denied
...
Order allow,deny
Deny from all
Satisfy All

They are for protecting data folders in DokuWiki, specifically for Apache older than version 2.4. The config is copied directly from their official documentation. Then I noticed another few lines:

<LocationMatch "(data|conf|bin|inc|vendor)/">
    Order allow,deny
    Deny from all
    Satisfy All
</LocationMatch>

The “vendor” regex keyword immediately caught my eyes. Recall that the forbidden JavaScript file is placed under somewhat “/vendor/” folder. Also, this <LocationMatch> section lays at the top level without scope restriction, meaning it will be applied by Apache to the whole server, even though it is only configured within one site.

It’s easy to fix the problem if knowing the root cause. By adding a prefix in the match regex, the forbidden error in WordPress went away and the admin page is again working like a charm.

<LocationMatch "/wiki/(data|conf|bin|inc|vendor)/">
    Order allow,deny
    Deny from all
    Satisfy All
</LocationMatch>

It is also possible to fix this by adding new directives introduced from Apache 2.4, without using <LocationMatch>:

<Directory /usr/share/dokuwiki/data>
        Require all denied
        ## FOR VER APACHE2.4

        ## Add other directories one by one
</Directory>

Hope this article can save someone a bit of time. It took me almost 2 hours to figure out what was going on. Have a good day!

The post Debugging “403 Forbidden” Error Accessing Files on Apache 2 appeared first on weijie ou's scratch pad.

Intro

Last year (2021), I decided to move on from Microsoft, where I worked in the past 4 years. Microsoft is absolutely one of the best workplaces for Software Engineers. Azure is even better. At Service Bus team, I learnt a ton on how to build and maintain a stable cloud service which serves millions of customers including large enterprises.

Interviewing sessions took about 3 months, and Discord is the last company I interviewed with. At the time Yen, our Director of Data Platform, reached out to me directly on LinkedIn, I was almost going to accept the offer from Square. Luckily, I still finished the process with Discord and found it most attractive.

To be honest though, I was not a regular Discord user before I joined. However, it was a significant factor for my decision that the app is well-designed, super snap and super user-friendly. The UI/UX and feeling was impressive the first time I dived into the app. Not to say the voice and video streaming quality is the best among all the online meeting/chatting app that I have used for prolong period. Teams (yes as a ex-Microsoft-er I do think Teams is doing great but can be much better) and Zoom can really learn from Discord, in my own opinion.

As an Engineer

So far I have no issue or complaints about the engineering system at Discord. Like most start-ups, the engineering process is flexible and fast-pace here. Unlike most start-ups, the system and toolset are also mature enough for engineers to not worry too much about anything outside the product they build, for example the infrastructure, CI, and deployment.

The development setup is straightforward, effortless and compatible. I easily initiated local dev environment on all my machines in both MacOS and Windows (via WLS2 Ubuntu). My favorite editor Visual Studio Code is by default supported with basic workspace and settings files integrated in the repository. Spinning up the local environment and get read for coding is as simple as typing one command.

Managers are supportive, engineers have great control on the design and code. Passionate people are contributing (a lot) to the repo everyday, progressing the app and service. Everything is moving fast, but right things are being done. Check-ins following good engineering principles and reviews are neat.

Stale components are also retired properly by removing relevant code and references. Engineers are encouraged to spend time on housekeeping works which I think is fantastic to keep the code base always in high quality and clean state.

Collaboration

This section is observing not only form an engineer’s perspective. Internally we heavily use Discord as the communication tool (for sure!), which is a new experience to me. We all know that collaboration nicely happen when people are actively communicating. Discord is doing a great job to replace even emails, with good ability of searching chat history.

Real-time DM tool like Discord has an advantage that you usually don’t have to wait long before people reply. For emails, there could be days of waiting time and “buried in piles of email” is a common reason that you don’t get prompt response.

It is also a breeze to start voice conversations with whoever you want to talk to, and stream your screen with almost no glitches. Of course, you are very welcomed to subscribe to Nitro for high resolution streaming that greatly improves the experience of your audiences.

From the experience of leading my first ground zero project at Experimentation Team at Discord, I like how my colleagues are friendly, participating and contributing to my own project. I received tons of useful feedbacks on scoping, design and coding from engineers, data scientists and PMs. They are proactive and read through the documents carefully, giving considerate comments that you can really apply to improve the overall quality of your project.

Meetings are effective, too. While I don’t currently have too many meetings on my calendar, every session I attended or led, is super fruitful. We tend to get prepared for the meetings, even with pre-read minutes so that everyone will be on the same page. Attendees share their opinions and concerns out of thoughtful observations. You can certainly get valuable feedbacks rather than wasting 30-60 minutes just walking through PPTs.

I feel that everyone is doing the best to share his/her knowledge and perspective to make others more productive. So that the project can further progress and as a result, Discord products are becoming better and better. I really do love this kind of collaboration.

Culture

I love the culture at Discord, it is inclusive, active and belonging. It might be still early and I have more to experience. Many employees are gamers, which I knew from the beginning since I was so surprise to find one interviewer know the title “428: Shibuya Scramble”, a game that not even known by many AVG players. People are talking like friends, discussing and collaborating like comrades.

Discord has a shallow top-bottom structure in regards of leveling. My most impressive new-hire experience is that Jason Citron, our CEO, will hold an hour-long meeting with all the new employees in that week to share the history and outlook of the company. He also welcome any question and be very careful, concrete and specific on the answers. I appreciate Jason for doing a bit research before he coming back to my question via direct message, I felt respected as a member of the Discordians.

These all make up to what Discord advocates to the world: build belongings. While Discord is striving to make here the best place for any group or community, we as the builders feel strong belonging to the company as well. I truly believe that we have the right direction and strategy as a company, and the future has more to come. Discord is playing a very important role to accommodate more and more people to provide open, smooth and reliable experience for all sorts of communication. In the past two years, pandemic brings increasing demand for the community to find a belonging place in the virtual world, and Discord is the first and best for that, ever.

The post First 3 months at Discord as a Software Engineer appeared first on weijie ou's scratch pad.

Spring 2021 was my last semester in the master program at Georgia Tech. I chose to push my limit and elected two courses:

• 6515 Graduate Introduction to Algorithms
• 7210 Distributed Computing

The second course is new on OMSCS. Nobody has elected this one before, we are literally the first batch of students to test the course.

It turns out the course is excellent and my passion on distributed systems aligns with the contents very well.

Course Contents

Let’s first see what are included in syllabus:

• Introduction to Distributed Systems
• Primer of RPC
• Time in Distributed Systems
• State in Distributed Systems
• Consensus
• PAXOS and Friends
• Replication
• Fault-tolerance
• Distributed Transactions
• Consistency and Geo-Distributed Data Stores
• Peer-to-peer, Mobility
• Distributed Data Analytics
• Distributed Machine Learning
• Support for Datacenter-based Distributed Computing
• Byzantine Fault Tolerance, Blockchain
• Edge Computing, IoT

The course covers a large range of theory and real-life practices in distributed systems. While it starts from fundamentals, it also goes up to the latest edge development of the industry, including graduate level topics like consensus, replication and fault-tolerance etc.

For the suggested readings, you can find classical articles and industry showcases from the companies running the largest distributed (and geo-distributed) systems in the world. Some examples:

• Raynal and M. Singhal. Logical Time: A Way to Capture Causality in Distributed Systems (Links to an external site.). IRISA Technical Report. (up to Section 7)
• Lamport, Time, Clocks and The Ordering of Events in Distributed Systems
• Spanner: Google’s Globally-Distributed Database
• Scaling Memcache at Facebook
• Gaia: Geo-Distributed Machine Learning Approaching LAN Speeds

One of the hottest topics – machine learning – is also adapted in this course in a DS fashion. You will learn in a geo-distributed scenario, what are the advanced ways to make sure the learning system is working at most efficiency, utilizing compute power globally.

Projects

Well, I would say this course has the most hardcore projects among all the ones I studied in the program. It is even much harder than those in 8803 Compiler. You will be asked to implement a bunch of things which are critical components/concepts of a distributed system:

• Key-value stores
• PAXOS consensus
• View server
• State transition and consistency in distributed system
• Two-phase commit
• Distributed transaction
• Async message delivery in distributed system

Projects are kind of related as one would be the base of another, especially for project 4 and 5.

The only complaint I have is the instructions, which can be improved to state the project intent and requirements much better. Most time I struggle understanding what should be done and what is the expected way to do it.

There are a bunch of tests running on GradeScope to evaluate your implementation. Two types of tests to expect: The RUN tests and SEARCH tests. Search test is a relatively new concept to me. The test code is not checking if the input/output are valid, but walking through the STATEs that your program can reach and see if any invariant violation. The concept reminds me something learnt in CS6340 Software Analysis and Test, which tests for invariants of code block, loops and branches.

The search test can be very difficult to pass reliably if the logic of the implementation is not or near to perfect. Any edge case, including super trivial ones, will be caught when the test walks into a state that is violating any assumption. Compare to run tests, search tests is very efficient in finding missed edge cases, along with the DSLab framework, to wrap everything in-memory (including I/O operations).

Difficulty of projects increases tremendously after Project 3 (as of Spring 2021). The first two are quite straightforward and simple, if you have some exposure to software engineering. You implement a basic key-value store, extending the provided interface and base Application class. After that, PAXOS comes into play and you are going to scratch (to the ground) your head for over 50 hours to implement the famous consensus protocol from Lamport’s thesis Paxos Made Simple. Well it feels like the “How to draw an owl meme”:

For sure, you will learn A LOT at the end, when you finish all five projects. It’s okay if you cannot get full score in all of them, I did not in the last one. Still, learn by coding is a good approach for studying distributed system, and computer science.

Conclusion

By striving through the course, I firmly believe that my passion is with Distributed Systems and I want to continue this journey. Go deep and go wide, that’s what I will be trying to do.

I strongly recommend anyone who is interested in the topics to take this class, it is well-organized, content rich, with good support from the professor and TAs. The topics are not stale, you could even leverage most of things you learnt in your job, if you work in the cloud industry.

Also applied to be an TA in this course, see you in Fall 2021.

The post Reflection for My Last Semester at OMSCS appeared first on weijie ou's scratch pad.

Problem:

There are n bulbs that are initially off. You first turn on all the bulbs. Then, you turn off every second bulb. On the third round, you toggle every third bulb (turning on if it’s off or turning off if it’s on). For the ith round, you toggle every i bulb. For the nth round, you only toggle the last bulb. Find how many bulbs are on after n rounds.

You can find the problem at https://leetcode.com/problems/bulb-switcher/.

I’m not going to talk about how to actually code the problem here, just want to share how I thought and finally got to the best solution.

The very naive way would be, create an array of length n, then loop through the array n times to switch the bulbs. This will work, but obviously comes with a O(n^2) time complexity. For that, Leetcode will not pass your solution, and we don’t like an algorithm at O(n^2). OK, then there must be something nicer and quicker to do the job. Let’s think about it.

Assume n is big enough, at the first round, we switch all bulbs:

[On, On, On, … , On]

At the second round ,we switch every two bulbs:

[On, Off, On, Off, … , On] (if n is an odd number) [On, Off, On, Off, … , Off] (if n is an even number)

At the third round, we switch every three bulbs:

[On, Off, Off, Off, On, On, On, … , Off] (if n is an odd number and it could be divided by 3) [On, Off, Off, Off, On, On, On, … , On] (if n is an odd number and it could not be divided by 3) [On, Off, Off, Off, On, On, On, … , On] (if n is an even number and it could be divided by 3) [On, Off, Off, Off, On, On, On, … , Off] (if n is an even number and it could not be divided by 3)

Hmmm, cannot get any pattern till now. But we can see, as the count of round goes up, the former part of array will not be touched anymore. Let’s give a position number to all the bulbs, starting from 1 to n.

[1, 2, 3, …, n]

And switch the words one, two, three to number 1, 2, 3 … n. OK, so before the round reaches a particular number (bulb), it could be switched one or more times. After that, it will remain the same status. How can we know how many times will a bulb be switched? A straight idea is to find its divisors.

It is easy to understand that, in this situation, a number (bulb) will only be reached when the count of round is a divisor of it. You could also call it prime factors. For example:

• Bulb at position 6 will be switched at 1st, 2nd, 3rd and 6th round.
• Bulb at position 69 will be switched at 1st, 3rd 23th and 69th round.

And so on! Then we know how to calculate the times of a given number is reached. Suppose we have a good algorithm to get all divisors of a number, we can get the solution to the number much more quicker. From this chart, it seems like a number k will have much less number of divisors: https://www.math.upenn.edu/~deturck/m170/wk2/divisors.html. Maybe the average time complexity will decrease to less than O(n/2)? You loop through all numbers and see if the bulb at this position will be on at last. The code looks like:

However, this solution didn’t pass because it is using too much time. So I had to re-think about the algorithm I applied. I thought it about an hour (yes, very long time) and suddenly noticed that the divisors are all in pairs! This is really an a-ha moment for me when solving this problem. Let’s see:

• Position 6: divisors are [1, 6], [2, 3]
• Position 69: divisors are [1, 69], [3, 23]

Did you think of anything? When a bulb is toggled twice, it will be off. So the bulb at #6 and #69 will be off at last. They will not be counted into the number of bulbs are on. But there must be some bulbs remain on when the loop is ended. Which numbers? From the above examples, we can see at least position #1 is on. But why it is one? Because it only have one divisor “1”. Just think a little bit deeper, or you have listed enough examples to observe the divisor pairs, you will find out:

• Position 9: divisors are [1, 9], [3, 3]
• Position 16: divisors are [1, 16], [2, 8], [4, 4]

A-ha! A bulb at position with a number which is perfect square will survive! That’s the critical point. If you still not getting, let me explain it.

We are looping all the bulbs, skipping one more bulb after every round; Consider every position has its number, it will only be reached when the round number is a divisor of its position number; Divisor appears in pairs, unless the divisor is a square root of the position number; A square root will only be reached once, bulb will be toggled only once at [square root]th round. At all the other rounds, the bulb will be toggled on-then-off.

The conclusion is, only the bulbs at a position which order number is a perfect square, will the bulb be on at last. We just need to find out how many perfect squares are there in range [1, n] (inclusive). The code will look like:

def bulbSwitch(self, n): """ :type n: int :rtype: int """ count = 0 i = 1 while i < n + 1: if i ** 2

Why stop when i ** 2 greater than n? Because you have already found the square roots of all the perfect squares within the range, no need to loop further. So someone even gave better solution: just return int(math.sqrt(n)) and this will be the answer.

That’s it. Please leave any comment if you think I’m wrong or you have suggestions for my expressions (as I’m not a native English speaker, just want to make myself more clear on explaining an algorithm problem).

The post Summary and thinking on Bulb Switcher problem appeared first on weijie ou's scratch pad.

When you compare two floats, you’d better not directly compare them. Because in the calculation handled by computer, some minor difference will be introduced into infinities. Try to compare floats if they are equal using:

Math.abs(1 / 3 - (1 - 2 / 3)) < 0.0000001; // true

Convert data to a particular format

toLocaleString() is good for converting a number to formatted string. You do not need to implement any extra methods yourself, JavaScript has done this for you. For example, you have some price to show in a table, you want to display it in USD currency format, just one line:

// Displays "128,000,000" if in U.S. English locale
var price = 128000000; console.log(price.toLocaleString());

Comparing with == and ===

Try to compare with === but not ==. == will convert the variable for comparison, === will only compare when the two comparators are the same type. Using === can avoid some unexpected behavior.

However, there is one exception. NaN === NaN will always return False. The only way to judge if a variable is to use isNaN()

Dynamic Binding

The event handlers can only bind to static elements if you use $(selector).eventHandler(...).

Try to use $(document).on('eventName', handler(){...}? to bind elements that are dynamically generated.

.next(selector)

Will only give you the next sibling, but not finding the next available element for you.

That is, if the next element is not matching selector, it will return empty. Without a selector, it will just give you the next element in DOM tree.

If you want to find anything in all the siblings, use .nextAll(selector) instead.

The post JavaScript (jQuery AngularJS) Notes appeared first on weijie ou's scratch pad.

Firstly, prepare your system for development and compilation:

sudo yum groupinstall "Development tools"

Please note that you might receive “Group Development tools does not exist” warning if you don’t have proper repositories. Try to install EPEL, CentOS-base or other necessary repositories. Also remember to setup proxies on if your server is in an internal network, otherwise yum will not work to get external resources.

Then, install the libraries required for compiling Python. As we are compiling a 32bit application on a 64bit system, we need to grab 32bit version of all these libraries. On CentOS 6, append ‘.i686’ to the package name for 32bit versions.

sudo yum install glibc-devel.i686 glibc-devel zlib-devel.i686 bzip2-devel.i686 openssl-devel.i686 ncurses-devel.i686 sqlite-devel.i686 readline-devel.i686 tk-devel.i686 gdbm-devel.i686 db4-devel.i686 libX11-devel.i686

Note that libX11 library will be installed by tkinter but not libX11-devel, so we have to install it separately. Now download satisfied version of Python from python.org. Let’s say put it in your home directory.

wget [https://www.python.org/ftp/python/2.7.12/Python-2.7.12.tar.xz](https://www.python.org/ftp/python/2.7.12/Python-2.7.12.tar.xz)

Unzip it:

unxz Python-2.7.12.tar.xz tar xf Python-2.7.12.tar

OK, we are set to start the compilation process. Change directory:

cd Python-2.7.12

Create a folder for your Python installation:

mkdir /opt/python-2.7.12.i686

Configure the compile:

BASECFLAGS=-m32 LDFLAGS=-m32 CFLAGS=-m32 ./configure --prefix=/opt/python-2.7.12.i686 --enable-shared

The -prefix option allows you to install Python in a custom location (instead of /usr/local); --enable-shared option tells make to build both static and dynamic Python libraries. The -m32 flags tells the compiler to use 32bit libraries for building. By default, gcc will use 64bit libraries on a 64bit platform.

Make the Python source files:

make

Search any library if the make failed on some modules (though it should not happen). You can ignore bsddb185 and sunaudiodev if you don’t need them. But other modules should be built for a complete functioning Python installation. From https://gist.github.com/reorx/4067217 we know these libraries are undocumented or deprecated:

• bsddb185: Older version of Oracle Berkeley DB. Undocumented. Install version 4.8 instead.
• dl: For 32-bit machines. Deprecated. Use ctypes instead.
• imageop: For 32-bit machines. Deprecated. Use PIL instead.
• sunaudiodev: For Sun hardware. Deprecated.
• _tkinter: For tkinter graphy library, unnecessary if you don’t develop tkinter programs.

Another consideraion is, there a bug in ffi.c in Python source code. When I was compiling, make returned an error message like this:

/opt/python-3.4.2/Python-3.4.2/Modules/_ctypes/libffi/src/x86/ffi.c:679: undefined reference to `ffi_closure_FASTCALL' /usr/bin/ld: build/temp.linux-x86_64-3.4/opt/python-3.4.2/Python-3.4.2/Modules/_ctypes/libffi/src/x86/ffi.o: relocation R_386_GOTOFF against undefined hidden symbol `ffi_closure_FASTCALL' can not be used when making a shared object

This will prevent you from successfully building _ctypes module. Through a little bit search, I found out this is caused by the mixed definition of calls for different platforms (x86 and x64). Some awesome guys provided a patch at https://bugs.python.org/issue23042. Read through the patch file (https://bugs.python.org/file39321/ffi.patch) and fix the logic in your ffi.c. In my case, I just need to move one line of #ifdef X86_WIN32 upper, before the line else if (cif->abi == FFI_FASTCALL) near line 678.

After make is succeed, install Python binary into the folder we created before:

make install altinstall

The altinstall option make sure the new installation will not replace the current version shipped with your system. Now let’s head to the installation folder and run the new built Python:

cd /opt/python-2.7.12.i686/bin ./python

You might encounter this error message when starting up:

libpython2.7.so.1.0: cannot open shared object file: No such file or directory

Modify your LD_LIBRARY_PATH, add /opt/python-2.7.12.i686/lib to this environment variable:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/python-2.7.12.i686/lib

You will want to save this export statement into your own bash configure file (usually ~/.bashrc) to make the change permanent. I also suggest to create a virtualenv for the new Python installation for further use.

REFERENCES

https://www.iram.fr/IRAMFR/GILDAS/doc/html/gildas-python-html/node36.html

http://shouro.blogspot.com/2013/07/compile-python27-on-centos-6.html

https://bugs.python.org/issue23042

The post Compile & install 32 bit Python 2.7 on 64 bit CentOS 6.5 appeared first on weijie ou's scratch pad.

Command Line Method

It is really easy to confirm a Python version using the option –version of the interpreter.

ares$ python --version Python 2.7.10
ares$ python3 --version Python 3.5.1

You can also start the interpreter, and see the version information in the welcome message.

ares$ python
Python 2.7.10 (default, Sep 23 2015, 04:34:21)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

Programmatically Check

You can also check the version on-the-fly when your program runs. Just a short piece of code:

import sys # if it is Python2.x is_py2=(sys.version_info[0] == 2) # if it is Python3.x is_py3=(sys.version_info[0] == 3)

By integrating the two values, you can check the current running environment and implement compatibility stuffs, such as __unicode__() and __str__() for Python 2.

Let’s take a close look at what sys.version_info returns:

>>> import sys
>>> sys.version_info
sys.version_info(major=2, minor=7, micro=10, releaselevel='final', serial=0)
>>>

The post How to check Python version (both command-line and programmatically) appeared first on weijie ou's scratch pad.