Last Saturday 12th November myself and eight of my teammates donned our iWeb hoodies, packed our rain macs and headed up the M6 / West Coast Main Line for the 3rd Mage Titans Manchester 2016 conference.
We had a good cross section of the team representing:
By the end of the day I think it’s fair to say we all felt in a similar state of mind.
But come Monday morning and a quick glance at #MageTitansMCR it was great to see the after-glow of the conference and the huge amount of energy and ideas emanating from the community.
What did we take out of it? For a start, lots of notes! So, in the interest of community contribution and collaboration, here they are. It’s a mixed bag of perspectives with notes from a few different team members but hopefully they serve to provide some further food for thought. If you’ve got anything to add or would like to ask a question, feel free to drop us a line on Twitter (handles included below).
Ben Marks – Guest Host
Notes from Andrew Pemberton / @p3mbo & Andrew McCombe / @euperia
Focusing on improving the M2 Developer Experience
“Magento 2 still has many dark corners”
Making the online training for Magento 2 free for the whole of December and January. Magento U will have video content.
Official docker container coming soon for Magento 2. This will be for Windows and Mac users alike. Get going quickly and easily.
Working on improving patch quality for Magento 1
Have been working with SnowDog to improve the front end on Magento 2, SASS based
Consistent feedback is that M2 is better than M1 once you’re over the initial learning curve
Changing approach to patching. Running through partners first before mainstream releases
Blackfire can hook into CI and start a profile and catch problems going into production.
Datadog: Centralise logging.
With Full Page Caching (FPC_ we should serve as much as possible from the cache, e.g Homepage. Obviously some things are personalised, e.g basket.
Areas such as mini-baskets will need caching separately. This is referred to as hole-punching.
Example module 1: Opening times
The first example was a block that displays the opening times on the footer of all pages.
Custom dates etc – uses a block and it used on every page on the site.
Ask yourself: How often should this block output change?
Example module 2: My Favorites
The second example a was a ‘My Favorites’ section that is personalised to each user.
Shows heart icon whenever you have a product that is favourited.
How does Magento handle cache validation of personalised data? Basket saves in a cached personalised container.
Include favourites in javascript with minimal data (for example, product ID’s only.). Add heart icon via JS after page load. Serve cached full page and render changes in browser.
One discovery was a crontab entry that cleared the cache every minute. The reasons for this are unknown, but you should share reasons – everyone is learning.
Magento 2 Full Page Caching with Varnish still has the same challenges as with Magento 1 – independently flushable content. Can use Varnish Edge Side Includes (ESI) to serve private content. Use AJAX and store in the browser’s localStorage.
Gotcha: If you use cacheable=false in a block xml it will invalidate the caching for entire page, and not use FPC. Don’t use $_isScopePRivate – is obsolete/deprecated.
A Magento 2 FPC Monitoring tool not available at the moment. Varnish is serving the full hits so we can’t use a Magento module for New Relic.
Final Thoughts:
Plan for FPC at the start of the project, monitor as much as possible, embrace the mistakes, and learn as much as possible.
Max discovered that not many people were looking into or blogging about Magento 2 front-end development.
You can do your own thing with Magento 2 front-end. If you can do that, why have a front-end framework at all? Magento loves the mom/pop shops. Some customers optimise for longevity rather than coolness.
SASS
Devs love SASS
Snow.dog is becoming a core initiative. SASS is likely to become core. Try using the Snow dog theme.
Fast Compilation
Core: measured in minutes, not seconds.
ES6 support:
ES6 is the next generation of the Javascript language, will become mainstream soon.
Currently compiled via Babel. Can be used with Grunt/Gulp.
Tradeoffs: build step is slower – but it is worth it.
Performance
Best practice: use AJAX for customer data, varnish everything else.
First rule of security: keep software updated.
Content Security Policy (CSP) An HTTP header that informs a browser to trust specific domains on page.
Goal: 100% CSP in Magento 2.
To do this we need to remove inline scripting in Magento 2
S: Single Responsibility – A class should only do one job O: open/closed. – objects should be extensible, but closed for modification: use interfaces, abstraction etc. L: Liskov substitution principle – “objects in a program should be replaceable with instances of their subtypes without altering the correctness of that program.” I : Interface Segregation A client should never be forced to implement. D: Dependency Inversion Principle.
Magento 2 is pretty good at this. Every time you see an interface there should be a concrete implementation.
“A Framework is one of the tools that helps you develop better or faster” – Symfony documentation.
Convenience v Maintainability: important to strike a balance.
Using Behat to generate acceptance xcriteria, written in Gherkin lang
Example showed building a Google product feed, using Behat to make it testable. The code enforced the SOLID principles using an example of extending the module to output xml as well as json.
Vitaly attended the previous day’s hackathon where he realised that a lot of the content in his planned talk was outdated or invalid. Therefore he modified his talk to:
“Building pretty much anything with Magento 2 Front-end components”
Feel free to update documentation if you feel you can improve it. It’s open source!
4400 words in US Constitution, 19,403 in Magento documentation
Forget about jQuery style element and class selectors. It’s not how it works anymore.
DOM is not immutable.
Isolation
M2 admin
Use xdebug/xhprof to learn what is happening with PHP. Use JS profiling tools to learn how the front-end works.
Don’t use PHP to render complex forms. Use UI components to render the forms via AJAX calls to get the data required.
Example is getting the grid in the product page in the admin. In Magento 1 it would submit all the fields from the product ‘toolbar’ form. In Magento 2 we only submit changed information.
No standard <form>tag anymore.
Interaction
Interaction in admin, i.e. select element option change can trigger an AJAX request to get new data – no page reloading.
UI components it uses configurable settings to handle form interactions. These are described on the devdocs.
There are improvements in the Javascript performance in M2, e.g iteration over list elements, pagination etc.
Form improvements (from >2.1.3 onwards), and grids (>2.0.0)
Extensibility
UI components are using a similar mindset to classic FE development
UI components on the server are not really up to the same standard as front-end
Use $.async to manipulate DOM within the asynchronous components.
Dedicated Mage backend servers for Magento workers – runs crons on separate instance
Building Blocks
AWS is fault intolerant. EC2 – server instances – provisioning
Use auto scaling groups for EC2. Not about auto-scaling traffic.
Actually about resilient auto-failing. i,e, one EC2 instance fails, it auto launches another.
Fully automated launch configuration
If you want only 1 instance – still put into auto scaling group. No more 3am wake ups!
Launch Configuration
Define the type:base AMI (region specific, OS) Provisioning script: ansible, puppet etc.
Bake your own AMI’s. Create your own, provision, and then ‘save’ it for future use. AKABaking
Immutable Deployments
Split the disposable parts from the non-disposable, i.e, web front ends are disposable, DB master isn’t.
Connecting the blocks
Loose coupling by using service discovery. Route 53 DNS using cnames, use AWSAPI, use tags.
Monitoring
Keep an eye on metrics.
Cloudwatch (Amazon), Datadog, New Relic
Be Prepared!! Try to anticipate any problem. If you can’t, work on recovering fast instead.
Stress limits
Know your infrastructure’s limits.
Learn and use Apache JMeter for testing.
Choose the right partners
Amazon Digital Ocean Payment providers CDNs etc
Verify that all the services and providers in your critical path
KISS! Keep It Stupidly Simple.
NIHS (Not invented here syndrome)
Don’t try to solve the problems yourself. Pay other services that can do the job for you, i.e New Relic etc.
Questions
How does Fabrizio handle switching to mysql master/slaves?
Use CNAME to switch to master/slave.
How do you rollback if something fails?
For critical CI deployments do not destroy previous build. Blue green deployment. Use N98 Magerun to deploy DB changes, via a Magento worker that processes updates/upgrades.
The Irish Store is a season business, doing 95% of their annual turnover around Christmas time. Their existing site was launched in 2013 and had 4,000 products (around 1,000 configurables) and over 500 categories.
About the talk
After 8 months of development, they have recently launched their new site on Magento 2. They wanted to upgrade to Magento 2 to improve all of the ‘touch points’ for their users (packaging, emails, website, tone of voice, brand).
The plan was to start the project in January 2016 and finish in August 2016. During the project the workload increased dramatically, the team also had to fix a lot of M2 bugs themselves. As the workload increased they used SCRUM and worked in 2 week sprints with daily stands up to get the basic functionality working with a plan to add features in later sprints. During the data import they found that they needed to exclude more columns than they needed to include so decided to do a manual data integration.
They installed 13 third party extensions but 1 only passed their internal QA (Realex Payments), the rest failed.
Their internal targets for the site were multi-availability, 99.9% uptime, 3 second page load time and able to handle < 300 orders per day.
They had to change their AWS (Amazon Web Service) hosting from 2 servers (1x site, 1x db) to 5 servers with load balancing. They also use tools like New Relic, Pingdom and Webpagetest to keep an eye on their Magento store’s performance.
They assume that Magento 2 releases will be every 3 months. Performance testing that they do is based around number of orders, as that is the main point of the site. They expect more features in Magento 2 and due to the documentation they expect the client/merchant to pick up and use these features. The infrastructure of a site should be thought about at the start of the development process, so you know what will need caching, etc.
Irish. Tadhg is a Magento Lead Developer at Screen Pages.
About the talk
Magento Enterprise client migrating to Magento 2. They have 21k+ customers, 5k+ products, 45+ orders, 100k+ quotes. The site had 15+ third party extensions and 40+ in-house extensions.
They used the built in Magento import tool to import the data across, he said the key was in the prep work, using third party extensions where possible and lots of dry runs.
After getting errors on the data import use the map.xml file to fix data.
The talk was about the importance of keeping Magento up-to-date. After the initial “warm up” Rebecca gave us some stats on Magento sites and what version they are running, worryingly there are 11 sites out there still running Magento 1.2!
Some agencies treat Magento sites like a factory production line, running sites off and then never updating or evolving those sites.
A new Magento site is a lot like having a new puppy (Max): they require daily attention, daily updates to content, products, banners and CTAs help with their SEO performance and help with keeping customers engaged. It is also import to keep the hosting platform up-to-date to allow Magento to perform to it’s full ability.
Training merchants to use the platform is also crucial in getting the most out of Magento. When updates and patches are released, we should make sure that these are rolled out and not ignored. Max 2 promises to make this process less painful with the updates on the second to last slide.
Take away: give your Magento plenty of love, care, and attention to get the best out of it.
Magento 1 API import was really slow. Magmi import can throw up errors. The best way to import data was through the Import/Export feature in the admin, importing data via a CSV.
Magento 2 uses the same SOAP/REST API but it is slow, the Import/Export feature is still the better way to import but it’s not automatic. It does now support external images though. There are 81 fixed columns in the export, no matter what data the user decides to export. Multiple attributes are exported in a single column, delimited by a vertical bar with the values of that attribute delimited by a comma.
Jason is Vice President of Technology at Magento. He is responsible for engineering, quality, operations, PMO, user experience, design and technical documentation. If you find a bug in Magento it’s his fault! He is “rubbish on Twitter”.
About the talk
Jason talked about the community engagement from Magento and the need for it to improve. He acknowledged the excellent community that Magento has, which is backed up by the 6,000 contributions to Magento 2 through github.
When they launched Magento 2 Merchant Beta should have been non-production even though it was put on some production environments, this lead to Magento 2.1 being an RC (Release Candidate) and was specifically for development purposes only, non-production! Magento have also got the Magento 2 B2B to the Alpha stage now, so they are looking to work closer with agencies and merchants. It is the first time Magento have released code before the MVP product. They are hoping to release code before all features are completed, this would be for non-production deploy so that extension developers can see the majority of the core updates and work with them, without waiting for the full official release, it gives them a headstart.
Jason said that Magento have learnt lessons from the SUPEE-8788 release and are looking to work closer with the community.
If you need a further fix of Mage Titans, check out their YouTube channel.
It was also great to hear that Mage Titans have raised £5,300 for worthy causes to date, with this year cause being the Royal Manchester Children’s Hospital.
Roll on 2017 when Titans continues the roadshow in Milan, USA, Mexico and Manchester once more.