development

Community contributions to MariaDB

One of the goals of the MariaDB Foundation is to help new contributors understand the source code and to lower the barrier for new participants. One way to measure this is to look at the number of pull requests received and accepted, as these mostly reflect community contributions. The figures below are for the main […]
The post Community contributions to MariaDB appeared first on MariaDB.org.

Observations on the hashicorp/raft library, and notes on RDBMS

The hashicorp/raft library is a Go library to provide consensus via Raft protocol implementation. It is the underlying library behind Hashicorp’s Consul.
I’ve had the opportunity to work with this library a couple projects, namely freno and orchestrator. Here are a few observations on working with this library:

TL;DR on Raft: a group communication protocol; multiple nodes communicate, elect a leader. A leader leads a consensus (any subgroup of more than half the nodes of the original group, or hopefully all of them). Nodes may leave and rejoin, and will remain consistent with consensus.
The hashicorp/raft library is an implementation of the Raft protocol. There are other implementations, and different implementations support different features.
The most basic premise is leader election. This is pretty straightforward to implement; you set up nodes to communicate to each other, and they elect a leader. You may query for the leader identity via Leader(), VerifyLeader(), or observing LeaderCh.
You have no control over the identity of the leader. You cannot “prefer” one node to be the leader. You cannot grab leadership from an elected leader, and you cannot demote a leader unless by killing it.
The next premise is gossip, sending messages between the raft nodes. With hashicorp/raft, only the leader may send messages to the group. This is done via the Apply() function.
Messages are nothing but blobs. Your app encodes the messages into []byte and ships it via raft. Receiving ends need to decode the bytes into a meaningful message.
You will check the result of Apply(), an ApplyFuture. The call to Error() will wait for consensus.
Just what is a message consensus? It’s a guarantee that the consensus of nodes has received and registered the message.
Messages form the raft log.
Messages are guaranteed to be handled in-order across all nodes.
The leader is satisfied when the followers receive the messages/log, but it cares not for their interpretation of the log.
The leader does not collect the output, or return value, of the followers applying of the log.
Consequently, your followers may not abort the message. They may not cast an opinion. They must adhere to the instruction received from the leader.

hashicorp/raft uses either an LMDB-based store or BoltDB for persisting your messages. Both are transactional stores.
Messages are expected to be idempotent: a node that, say, happens to restart, will request to join back the consensus (or to form a consensus with some other node). To do that, it will have to reapply historical messages that it may have applied in the past.
Number of messages (log entries) will grow infinitely. Snapshots are taken so as to truncate the log history. You will implement the snapshot dump & load.
A snapshot includes the log index up to which it covers.
Upon startup, your node will look for the most recent snapshot. It will read it, then resume replication from the aforementioned log index.

hashicorp/raft provides a file-system based snapshot implementation.

One of my use cases is completely satisfied with the existing implementations of BoltDB and of the filesystem snapshot.
However in another (orchestrator), my app stores its state in a relational backend. To that effect, I’ve modified the logstore and snapshot store. I’m using either MySQL or sqlite as backend stores for my app. How does that affect my raft use?

My backend RDBMS is the de-facto state of my orchestrator app. Anything written to this DB is persisted and durable.
When orchestrator applies a raft log/message, it runs some app logic which ends with a write to the backend DB. At that time, the raft log is effectively not required anymore to persist. I care not for the history of logs.
Moreover, I care not for snapshotting. To elaborate, I care not for snapshot data. My backend RDBMS is the snapshot data.
Since I’m running a RDBMS, I find BoltDB to be wasteful, an additional transaction store on top a transaction store I already have.
Likewise, the filesystem snapshots are yet another form of store.
Log Store (including Stable Store) are easily re-implemented on top of RDBMS. The log is a classic relational entity.
Snapshot is also implemented on top of RDBMS,  however I only care for the snapshot metadata (what log entry is covered by a snapshot) and completely discard storing/loading snapshot state or content.
With all these in place, I have a single entity that defines:

What my data looks like
Where my node fares in the group gossip

A single RDBMS restore returns a dataset that will catch up with raft log correctly. However my restore window is limited by the number of snapshots I store and their frequency.

Duel: gdb vs. linked lists, trees, and hash tables

My first encounter with the gdb command duel was on some old IRIX about 15 years ago. I immediately loved how convenient it was for displaying various data structures during MySQL debugging, and I wished Linux had something similar. Later I found out that Duel was not something IRIX specific, but a public domain patch […]
The post Duel: gdb vs. linked lists, trees, and hash tables appeared first on MariaDB.org.

Making life prettier with gdb PrettyPrinting API

Anyone who has peeked inside a gdb manual knows that gdb has some kind of Python API. And anyone who has skimmed through has seen something called “Pretty Printing” that supposedly tells gdb how to print complex data structures in a nice and readable way. Well, at least I have seen that, but I’ve never […]
The post Making life prettier with gdb PrettyPrinting API appeared first on MariaDB.org.

Percona Blog Poll Results: What Programming Languages Are You Using for Backend Development?

In this blog we’ll look at the results from Percona’s blog poll on what programming languages you’re using for backend development.
Late last year we started a poll on what backend programming languages are being used by the open source community. The three components of the backend – server, application, and database – are what makes a website or application work. Below are the results of Percona’s poll on backend programming languages in use by the community:
Note: There is a poll embedded within this post, please visit the site to participate in this post’s poll.
One of the best-known and earliest web service stacks is the LAMP stack, which spelled out refers to Linux, Apache, MySQL and PHP/Perl/Python. We can see that this early model is still popular when it comes to the backend.
PHP still remains a very common choice for a backend programming language, with Python moving up the list as well. Perl seems to be fading in popularity, despite being used a lot in the MySQL world.
Java is also showing signs of strength, demonstrating the strides MySQL is making in enterprise applications. We can also see JavaScript is increasingly getting used not only as a front-end programming language, but also as back-end language with the Node.JS framework.
Finally, Go is a language to look out for. Go is an open source programming language created by Google. It first appeared in 2009, and is already more popular than Perl or Ruby according to this poll.
Thanks to the community for participating in our poll. You can take our latest poll on what database engine are you using to store time series data here. 

Percona Blog Poll: What Programming Languages are You Using for Backend Development?

Take Percona’s blog poll on what programming languages you’re using for backend development.
While customers and users focus and interact with applications and websites, these are really just the tip of the iceberg for the whole end-to-end system that allows applications to run. The backend is what makes a website or application work. The backend has three parts to it: server, application, and database. A backend operation can be a web application communicating with the server to make a change in a database stored on a server. Technologies like PHP, Ruby, Python, and others are the ones backend programmers use to make this communication work smoothly, allowing the customer to purchase his or her ticket with ease.
Backend programmers might not get a lot of credit, but they are the ones that design, maintain and repair the machinery that powers a system.
Please take a few seconds and answer the following poll on backend programming languages. Which are you using? Help the community learn what languages help solve critical database issues. Please select from one to six languages as they apply to your environment.
If you’re using other languages, or have specific issues, feel free to comment below. We’ll post a follow-up blog with the results!
Note: There is a poll embedded within this post, please visit the site to participate in this post’s poll.

Why Get Java SE8 Certified?

Getting certified may not ever make it on
your ‘to-do’ list but it should.

Here’s
why:

  1. Being certified demonstrates a skill set that can set you
    apart from others
  2. These skills are in high-demand
  3. There are 37,981 current job openings in the United
    States 
  4. Of these 28,025 have a listed salary range of $100,000 or
    more
  5. Top employers are Amazon, Oracle, Accenture, CyberCoders and
    Capital One

 See for yourself. Click to see Skills, What it Takes to Earn
this Badge
and Related Jobs.

You can filter these real-time Labor Market Insights for India,
UK and Canada in addition to the United States.

And, if you certify by December 31st you get a Free
T-shirt too!

TEL/電話+86 13764045638
Email service@parnassusdata.com
QQ 47079569