I have developed a queueing library for Ruby that uses PostgreSQL as the backend. With the combination of row-level locking, PUB/SUB and PL/pgSQL, PostgreSQL was a perfect fit for this database backed queue. In the talk, I will highlight the motivations and pitfalls I discovered while building and maintaining Queue Classic. I will also discuss the environments in which the Queue Classic is running. Queue Classic supports a variety of components: runtime, database provisioning & financial services.
While Bucardo is often known for being one of the few multi-master replication systems available for PostgreSQL, there are also many other uses for it as a general tool for:
* multi-master configuration
* live/minimal downtime database upgrades
* live/minimal server migrations
* data aggregator
Features of Bucardo will be examined, as well as an architectural overview. We will also review its real-life use in a live migration from an existing multi-node Slony/PostgreSQL 8.2 cluster in one data center to a PostgreSQL 9.0/Hot Standby cluster in another with minutes of downtime. Comparisons will be made between Bucardo's approach and Slony's, and brief coverage of the options these differences open up will be made.
No existing familiarity with Bucardo is necessary, but knowledge of various existing replication systems and concepts will be a plus.
We will also cover the upcoming Bucardo 5.
This talk will discuss the impacts and implementation of fast, automated provisioning of replicants -- both as writable 'forked' copies and read-only streaming replica 'followers' of PostgreSQL databases, as implemented at Heroku on a large scale as a service.
Included will be a brief summary of use cases, anecdotes, and challenges, as well as new trade-offs in the design of applications and partitioning of data. In addition, the technical architecture behind such a service and how it unifies rock-solid, low-impact disaster recovery, replication, and database-forking will be covered.
TransLattice has developed new technology allowing geographically distributed, parallel databases. Starting with PostgreSQL technology and the open-source Postgres-R synchronous multimaster replication system, Translattice has extended the relational database with transparent horizontal sharding and redundancy mechanisms.
In this talk, TransLattice’s use of PostgreSQL will be discussed, along with TransLattice’s open source strategy. A brief overview of the associated replication, data partitioning, and coherency approaches will also be given within the TransLattice Application Platform.
The audience will gain insights regarding:
• Horizontally scaling deployments of RDBMS and PostgreSQL
• The Postgres-R synchronous multi-master replication system
• TransLattice’s approaches within the PostgreSQL ecosystem.
Powerful datatypes such as hstore and json give PostgreSQL the most compelling feature of many NoSQL databases, which is management of semi structured data. Unfortunately the popular Ruby ORMs try very hard to equalize all databases, so these and other features often go unnoticed or underappreciated.
While there are some Ruby libraries providing application-level bindings for these features, they are generally new or otherwise not well-known. Developers are also often further discouraged by having to sacrifice the benefits of their ORM to use these libraries. Fortunately, the situation is starting to improve.
We will go into detail about what was needed to bring hstore support to Ruby, survey the current PostgreSQL and Ruby landscape, and discuss what needs to happen in the near future to improve their relationship.
Application deployment has become increasingly complicated with today’s performance, scalability, reliability, and compliance requirements. Technologies, such as utility computing in the cloud, can help reduce costs but also introduce new challenges, including meeting application database and data storage requirements.
This talk includes a comparison of scaling and redundancy models. Attendees will learn how to maximize application resilience and performance while containing costs. In the current relational model, operations staff deals with the complexity of where data is located and how to scale up. In the do-it-yourself/big data model, the development staff does. New architectures are emerging that hand the responsibility to the infrastructure itself.
The audience will gain insights regarding:
• Realizing the benefits of Cloud Computing
o Scalability (Elasticity)
o Availability and Protection Against Data Loss
• Deployment models – Scale Up vs. Scale Out
• Conventional Transactional Infrastructure
• Parallel Databases for Analytics
• Specialized Architectures for Large Scale: NoSQL and Friends
• Emerging Efforts at Horizontal Scalability of Enterprise Application Stacks
Compute bound PostgreSQL Data Analytics and Data Mining queries can exhibit poor performance; as PostgreSQLs’ “per process model” constrains a query to a single CPU Core. With PgOpenCL Time Series, a new data type, compute intensive queries are parallelized to take advantage of all CPU cores. If a discrete GPU, or AMD Fusion GPU, is available, queries can take advantage of the additional 100’s to 1000’s of cores for further speed ups.
The Time Series data types are specialized data types that represent a sequence of data points measured typically at successive times spaced at uniform time intervals. Some typical examples are Wall Street closing prices, climate data and digitized signals. The PgOpenCL Time Series data types utilize native OpenCL data types to take maximum advantage of GPU hardware.
PgOpenCL is a new PostgeSQL Stored Procedure language that implements Open Computer Language (OpenCL) for Parallel Processing on Multi-Core and Many-Core systems. OpenCL is freely available on Windows and Linux systems.
Multi-version concurrency control is a well-known and established key technology for high performance database systems. Many open source projects, as well as commercial products, offer various successful implementations of MVCC. However, its potential for clustering solutions is still underestimated. TransLattice Application Platform (TAP) leverages MVCC to be used in an automatically sharded, multi-master cluster.
The audience will gain insights into:
The key features of Postgres-R and TAP, including:
• The concurrent, optimistic application of change sets
• Conflict detection during normal operation
• Distributed cross-shard operations
• Runtime healing in a multi-master environment