LHC Tunnel

LHC Tunnel

Tuesday 19 April 2016

Deploying the new OpenStack EC2 API project

OpenStack has supported a subset of the EC2 API since the start of the project. This was originally built in to Nova directly. At CERN, we use this for a number of use cases where the experiments are running across both the on-premise and AWS clouds and would like a consistent API. A typical example of this is the HTCondor batch system which can instantiate new workers according to demand in the queue on the target cloud.

With the Kilo release, this function was deprecated and has been removed in Mitaka. The functionality is now provided by the new ec2-api project which uses the public Nova APIs to provide an EC2 compatible interface.

Given that CERN has the goal to upgrade to the latest OpenStack release in the production cloud before the next release is available, a migration to the ec2-api project was required before the deployment of Mitaka, due to be deployed at CERN in 2H 2016.

The EC2 API project was easy to set up using the underlying information from Nova and a small database which is used to store some EC2 specific information such as tags.

As described in Subbu's blog, there are many parts needed before for an OpenStack API to become a service. By deploying using the CERN cloud, many aspects on identity, capacity planning, log handling, onboarding are covered by the existing infrastructure.


From the CERN perspective, the key functions we need in addition to the code are
  • Packaging - we work with the RDO distribution and the OpenStack RPM-Packaging project to produce a package for installation on our CentOS 7 controllers.
  • Configuration - Puppet provides us the configuration management for the CERN cloud. We are currently merging the CERN Puppet EC2 API modules to the puppet-ec2api project. The initial patch is now in review.
  • Monitoring - each new project has a set of daemons to make sure are running smoothly. These have to be integrated into the site monitoring system.
  • Performance - we use the OpenStack Rally project to continuously run functionality and performance tests, simulating a user. The EC2 support has been added in this review.
The current steps are the end user testing and migration from the current service. Given that the ec2-api project can be run on a different port, the two services can be run in parallel for testing. Horizon would need to be modified to change the EC2 endpoint in the ec2rc.sh  (which is downloaded from Compute->Account & Security->API Access).

So far, the tests have been positive and further validation will be performed over the next few months to make sure that the migration has completed so there is no impact on the Mitaka upgrade.

Acknowledgements

  • Wataru Takase (KEK) for his work on Rally
  • Marcos Fermin Lobo (CERN/Oviedo) for the packaging and configuration
  • Belmiro Moreira (CERN) for the necessary local CERN customisations in Nova
  • The folks from Cloudscaling/EMC for their implementation and support of the OpenStack EC2 API project

References





62 comments:

  1. It's ideal to discover different journalists share like personalities on some substance. This is the situation with your article. I truly delighted in this.


    SEO services in kolkata
    Best SEO services in kolkata
    SEO company in kolkata
    Best SEO company in kolkata
    Top SEO company in kolkata
    Top SEO services in kolkata
    SEO services in India
    SEO copmany in India

    ReplyDelete
  2. Whats Happening i am new to this, I stumbled upon this I’ve discovered It absolutely helpful and it has aided me out loads. I am hoping to give a contribution & assist other users like its helped me. Great job. ํ† ํ† 

    ReplyDelete
  3. It’s going to be end of mine day, however before end I am reading this wonderful article to increase my experience. ํ† ํ† 

    ReplyDelete
  4. Wow, awesome blog layout! How long have you been blogging for? you make blogging look easy. The overall look of your web site is great, as well as the content! ์ผ๋ณธ์•ผ๋™

    ReplyDelete
  5. Nice one! Thank you for sharing this post. ์นด์ง€๋…ธ์‚ฌ์ดํŠธ Your blog posts are more interesting and impressive.

    ReplyDelete
  6. It is very well written, and your points are well-expressed. I request you warmly, please, ๋ฐ”์นด๋ผ์‚ฌ์ดํŠธdon’t ever stop writing.

    ReplyDelete
  7. I finally found great post here. ํ† ํ† ์‚ฌ์ดํŠธ Thanks for the information. Please keep sharing more articles.

    ReplyDelete
  8. Simply desire to say your article is as amazing. The clearness on your post is simply spectacular and that์นด์ง€๋…ธ์‚ฌ์ดํŠธ i could assume you are a professional on this subject.

    ReplyDelete
  9. The filling rate of your site is superb. What did you do to enhance the filling rate of the web site? Please allow us understand, also. I assume internet sites with quick filling rate are an important part due to the fact that they are extremely practical for site visitors. I'll be awaiting your reply. I'll connect the web link. https://lampcasino.com

    ReplyDelete
  10. Hello there! I could have sworn Iรญve been to this web site before but after looking at many of the articles I realized itรญs new to me. Anyhow, Iรญm definitely delighted I came across it and Iรญll be bookmarking it and checking back regularly! ๋ฉ”์ด์ €๊ณต์›

    ReplyDelete
  11. Heya! I just wanted to ask if you ever have any trouble with hackers? My last blog (wordpress) was hacked and I ended up losing many months of hard work due to no back up. Do you have any solutions to protect against hackers?| ํ† ํ† ์‚ฌ์ดํŠธ

    ReplyDelete
  12. Hi there just wanted to give you a quick heads up. The text in your article seem to be running off the screen in Internet explorer.I’m not sure if this is a format issue or something to do with web browser compatibility but I thought ํŒŒ์›Œ๋ณผ์‚ฌ์ดํŠธ

    ReplyDelete
  13. Excellent blog here! Also your site a lot up fast! What web host are you the usage of? Can I get your affiliate link in your host? I wish my site loaded up as quickly as yours lo..Thanks for sharing such information with us due to which my several concepts have been cleared ์•ˆ์ „๋†€์ดํ„ฐ๋ชจ์Œ

    ReplyDelete
  14. Hello there! Would you mind if I share your blog with my zynga group?.There’s a lot of folks that I think would really enjoy..your content. Please let me know. Thank you ์šฐ๋ฆฌ์นด์ง€๋…ธ

    ReplyDelete
  15. Iรญm amazed, I have to admit. Seldom do I come across a blog thatรญs equally educative and amusing, and let me tell you, you’ve hit the nail on the head. The problem is something that not enough people are speaking intelligently about. I am very happy I stumbled across this during my hunt for something regarding this. ์•ˆ์ „๋†€์ดํ„ฐ

    ReplyDelete
  16. Hello there! Would you mind if I share your blog with my zynga group? There’s a lot of people that I think would really appreciate your content. Please let me know. Cheers|..I truly love this blog article.Thanks Again and again. Will read on ๋จนํŠ€๊ฒ€์ฆ

    ReplyDelete
  17. You have a good point here!I totally agree with what you have said!!Thanks for sharing your views...hope more people will read this article!!!//You have a good point here!I totally agree with what you have said!!Thanks for sharing your views...hope more people will read this article!!! ๋จนํŠ€๊ฒ€์ฆ

    ReplyDelete
  18. I like viewing web sites which comprehend the price of delivering the excellent useful resource free of charge. I truly adored reading your posting. Thank you! ..The Law Assignment Help we provide is flowered with great vocabulary without being unnecessarily verbose and that combined with the impressive presentation makes it every worth your while to seek us out. We are helping students globally for the last 6 years and looking forward for more. ๋จนํŠ€ํด๋ฆฌ์Šค

    ReplyDelete
  19. I found that site very usefull and this survey is very cirious, I ' ve never seen a blog that demand a survey for this actions, very curious..You have a good site. It is really easy, and nice to navigate.I think the website is great, looks wonderful and very easy to follow ๋จนํŠ€๊ฒ€์ฆ

    ReplyDelete
  20. I at last discovered grand post here.I will get back here. I just added your blog to my bookmark districts. thanks.Quality presents is the major on welcome the guests to visit the site page, that is the thing that this site page is giving.I see something fascinating about your website, so I saved to bookmarks. Web webpage: ๋จนํŠ€์‚ฌ์ดํŠธ

    ReplyDelete
  21. I can't consider centering enough long to inspect; incredibly less makes such an article. You've achieved something unprecedented for yourself with this material positively. It is maybe the best substance...I was fundamentally surveying along and staggered over your blog. Just expected to confer a decent blog and this article truly enabled me ์˜จ๋ผ์ธ์นด์ง€๋…ธ

    ReplyDelete
  22. I am thankful for the article post. Really looking forward to read more. Will read on...Superb brief which write-up solved the problem a lot. Say thank you We searching for your data...A very awesome blog post. We are really grateful for your blog post. You will find a lot of approaches after visiting your post. ๋จนํŠ€๊ฒ€์ฆ

    ReplyDelete
  23. I like viewing web sites which comprehend the price of delivering the excellent useful resource free of charge. I truly adored reading your posting. Thank you! ..The Law Assignment Help we provide is flowered with great vocabulary without being unnecessarily verbose and that combined with the impressive presentation makes it every worth your while to seek us out. We are helping students globally for the last 6 years and looking forward for more. ๋จนํŠ€๊ฒ€์ฆ

    ReplyDelete
  24. I found that site very usefull and this survey is very cirious, I ' ve never seen a blog that demand a survey for this actions, very curious..hi was just seeing if you minded a comment. i like your website and the thme you picked is super. I will be back. ๋จนํŠ€์‹ ๊ณ 

    ReplyDelete
  25. This is actually the kind of information I have been trying to find. Thank you for writing this information..Recognizes for paper such an advantageous creation, I lurched adjacent to your blog other than decode a restricted report. I need your system of engraving..Thanks for a very interesting blog. What else may I get that kind of info written in such a perfect approach? I’ve a undertaking that I am simply now operating on, and I have been at the look out for such info. ํ† ํ† ์‚ฌ์ดํŠธ

    ReplyDelete
  26. Newstyle Print offers high end, low cost banner printing in the UK.. like review goals which understand the cost of passing on the amazing strong asset futile out of pocket. I truly worshiped examining your posting. Grateful to you! ๋ฉ”์ด์ €๋†€์ดํ„ฐ

    ReplyDelete
  27. Hello! I could have sworn Iรญve visited this website before but after going through a few of the posts I realized itรญs new to me. Regardless, Iรญm certainly delighted I came across it and Iรญll be bookmarking it and checking back regularly! ํ”ผ๋ง๋จธ๋‹ˆ์ƒ

    ReplyDelete
  28. I am incapable of reading articles online very often, but I’m happy I did today. It is very well written, and your points are well-expressed. I request you warmly, please, don’t ever stop writing.Do you want to buy the best Chromebook for artists? Below we have picked out the top 9 to help you with your decision. Click the blue buttons for more info on Amazon.com ๋จนํŠ€๊ฒ€์ฆ

    ReplyDelete
  29. ove to read it,Waiting For More new Update and I Already Read your Recent Post its Great Thank..great article. thank you for sharin..Great post. Thank You For Sharing Valuable. information. It is Very Informative article..I urge you to peruse this content it is fun portrayed .. ๋ฉ”์ด์ €์‚ฌ์ดํŠธ

    ReplyDelete
  30. I’m very pleased to find this site. I need to to thank you for ones time due to this wonderful read!! I definitely really liked every little bit of it and i also have you saved as a favorite to check out new stuff on your site. Great post, I want to thank you for this informative read; I really appreciate sharing this great post. Keep up the good work! Very good written article. It will be supportive to anyone who utilizes it, including me. Keep doing what you are doing – can’r wait to read more posts. ํŒŒ์›Œ๋ณผ์‹ค์‹œ๊ฐ„

    ReplyDelete
  31. Quality content is the main to be a focus for the people to visit the web..page, that’s what this web site is providing.I am sure this paragraph has touched all the internet visitors,..its really really good piece of writing on building up new ํ† ํ† ์‚ฌ์ดํŠธ์ฝ”๋“œ

    ReplyDelete
  32. I read a article under the same title some time ago, but this articles quality is much, much better. How you do this.. Wonderful blog post. This is absolute magic from you! I have never seen a more wonderful post than this one. You've really made my day today with this. I hope you keep this up! You completed a number of nice points there. I did a search on the issue and found nearly all people will have the same opinion with your blog. ๋ฉ”์ด์ €๋†€์ดํ„ฐ

    ReplyDelete
  33. I am sure this article has touched all the internet people, its really really nice paragraph on building up new blog...These are really fantastic ideas in on the topic of blogging. You have touched some fastidious things here. Any way keep up wrinting. ๊ฝ๋จธ๋‹ˆ

    ReplyDelete
  34. This is such a great resource that you are providing and you give it away for free. I love seeing blog that understand the value of providing a quality resource for free . Wow! Such an amazing and helpful post this is. I really really love it. It's so good and so awesome. I am just amazed. I hope that you continue to do your work like this in the future also . Thanks for another wonderful post. Where else could anybody get that type of info in such an ideal way of writing? ํŒŒ์›Œ๋ณผ

    ReplyDelete
  35. Wonderful goods from you, man. I have understand your stuff previous to and you are just extremely fantastic. I really like what you have acquired here, really like what you are stating and the way in which you say it. You make it enjoyable and you still care for to keep it wise. I can not wait to read much more from you. This is really a tremendous web site. ๋จธ๋‹ˆ์ƒ

    ReplyDelete
  36. The article looks magnificent, but it would be beneficial if you can share more about the suchlike subjects in the future. Keep posting. ์Šฌ๋กฏ์‚ฌ์ดํŠธ

    ReplyDelete
  37. I truly like you're composing style, incredible data, thankyou for posting ๋ฉ”์ด์ €์‚ฌ์ดํŠธ ์ถ”์ฒœ

    ReplyDelete
  38. Just saying thanks will not just be sufficient, for the fantasti c lucidity in your writing. I will instantly grab your rss feed to stay informed of any updates ๋จนํŠ€๊ฒ€์ฆ

    ReplyDelete
  39. This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck. ํ™€๋ค

    ReplyDelete
  40. This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck. ํ† ํ† ์‚ฌ์ดํŠธ

    ReplyDelete
  41. Very useful post. This is my first time i visit here. I found so many interesting stuff in your blog especially its discussion. Really its great article. Keep it up. ํ† ํ† ์‚ฌ์ดํŠธ

    ReplyDelete
  42. An extremely magnificent blog entry. We are extremely thankful for your blog entry. battle, law authorization You will discover a ton of methodologies in the wake of going by your post. I was precisely looking for. A debt of gratitude is in order for such post and please keep it up. Awesome work ๋จนํŠ€

    ReplyDelete
  43. I can’t imagine focusing long enough to research; much less write this kind of article. You’ve outdone yourself with this material. This is great content. ์Šฌ๋กฏ์‚ฌ์ดํŠธ

    ReplyDelete
  44. Really impressed! Everything is very open and very clear clarification of issues. It contains truly facts. Your website is very valuable. Thanks for sharing ํ† ํ† 

    ReplyDelete
  45. The article looks magnificent, but it would be beneficial if you can share more about the suchlike subjects in the future. Keep posting. ํ† ํ† ์‚ฌ์ดํŠธ

    ReplyDelete
  46. The article looks magnificent, but it would be beneficial if you can share more about the suchlike subjects in the future. Keep posting. ํ† ํ† ์ปค๋ฎค๋‹ˆํ‹ฐ

    ReplyDelete
  47. I haven’t any word to appreciate this post.....Really i am impressed from this post....the person who create this post it was a great human..thanks for shared this with us. ๋ฐ”๋‹ค์ด์•ผ๊ธฐ

    ReplyDelete
  48. The article you have shared is very interesting. I am really happy that i visit your webpage, and i am agree with what you said. Please keep sharing more. ํ† ํ† ์‚ฌ์ดํŠธ

    ReplyDelete
  49. As educators, we are asking these students to engage with very traditional forms of literature whilst simultaneously producing narratives in a multimodal format that rely on various affordances of the internet. 
    Life Experience Degrees

    ReplyDelete
  50. It is important to note that my current students, whilst engaging with traditional books are also encouraged to publish their own work in multimodal forms using online editing tools such as TACKK or Shorthand Social.
    Life Experience Degree

    ReplyDelete
  51. I definitely need to challenge myself to be quiet more. I have forte a lot better but there is surely room for improvement! I feel like an old lady shaking my cane at a young whippersnapper when I say this, but what I wouldn’t give to have people in my real, every day, non virtual life who make it necessary not to spend all my time plugged in. ์Šฌ๋กฏ์‚ฌ์ดํŠธ

    ReplyDelete
  52. This blog made me realize how efficient stainless steel tube fittings can be, never know they were environment friendly as well. Definitely consider these points before changing the plumbing system.

    ReplyDelete
  53. Hey, thanks for the article post.Really looking forward to read more. Really Great.
    splunk online training
    sql server developer online training

    ReplyDelete
  54. All your hard work is much appreciated. This content data gives truly quality and unique information. I’m definitely going to look into it. ๋ฐ”์นด๋ผ์‚ฌ์ดํŠธ์ฃผ์†Œ

    ReplyDelete
  55. Long-term use of cigarettes will do harm to the lungs, and second-hand smoke will also affect the health of others, so it is advocated to ban the use of cigarettes. E-cigarettes can be used instead.Steam Crave RDTA and smok starter kit products are cheap, durable and healthier than cigarettes.

    ReplyDelete